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CHAPTER  1 

INTRODUCTION 

The  manner  in  which  people  interact  with  computers  underwent  a 
dramatic  change  in  the  1980's.  Before  1980,  small  computer  screens 
displayed  information  using  only  alphanumeric  characters  or  primitive 
graphics  constructed  from  letters,  numbers  and  S3rmbols.  Now,  large 
screen  bit-map  displays  present  information  in  many  forms  and  at  many 
locations  on  the  screen.  People  used  to  send  commands  to  the  computer  by 
typing  letter  combinations  or  words  memorized  from  a  command  language. 
Commands  are  recognized  and  selected  today  from  a  list  of  options 
presented  to  the  user  in  command  menus.  Increasingly,  the  selection  of 
information  does  not  require  keyboards,  but  involves  other  selection  devices, 
such  as  a  mouse,  trackball,  joystick,  or  touchscreen,  which  allow  direct 
manipulation  of  objects  on  the  screen. 

The  new  interaction  style  has  changed  many  of  the  psychological 
issues  that  are  important  for  understanding  and  modelling  human- 
computer  interaction.  The  old  style  emphasized  memory  processes  and 
keystrokes  associated  with  recalling  commands  and  entering  them  with  a 
keyboard.  The  new  interaction  style  places  a  much  greater  emphasis  on  the 
perceptual/motor  features  of  interaction,  locating  and  selecting  information 
on  the  screen. 

The  scope  of  relevant  motor-control  issues  has  also  been  greatly 
expanded  by  non-keyboard  selection  devices.  With  keyboard  entry,  the  only 
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significant  limb  movement  is  the  pressing  of  keys.  The  range  of  physical 
actions  required  by  the  new  selection  devices  is  much  broader,  including 
reaching  to  and  grasping  objects  that  are  in  different  spatial  locations, 
moving  the  selection  device  in  two  or  three  dimensional  space,  and 
coordinating  actions  for  selection  and  movement. 

The  range  of  possible  tasks  using  computer  technology  is  vast  and  is 
increasing  steadily.  One  challenge  for  conducting  research  in  this  field  is 
to  choose  a  small  set  of  tasks  that  representative  of  the  current  technology, 
and  that  will  be  relevant  to  future  developments. 

Menu-selection  with  a  mouse  incorporates  two  of  the  three 
innovations  in  interface  design,  namely  menu  displays,  and  spatial 
positioning,  that  Lewis  (1990)  identifies  as  central  for  the  decade  of  the 
1990's.  Menu-selection  has  become  a  primary  method  for  entering 
commands  on  new  and  emerging  computer  systems.  Menu-selection  is 
also  a  generic  task  independent  of  the  software  being  used,  so  the  findings 
of  this  research  should  have  general  applicability  to  a  wide  range  of 
situations. 


Overview  of  Dissertation 

This  dissertation  isolates  and  examines  some  of  the  emergent 
perceptual-motor  issues  raised  by  the  new  style  in  human- computer 
interaction.  It  concerns  the  use  of  a  mouse  to  select  commands  from 
menus.  The  rest  of  this  chapter  describes  the  physical  and  perceptual 
characteristics  of  the  menus  and  selection  procedures  to  be  studied  here. 
Chapter  2  then  reviews  research  from  both  the  motor-control  and  the 
human-computer  interaction  literature  that  applies  to  perceptual  and 
motor  aspects  of  menu  selection.  In  Chapter  2,  predictive  performance 
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models  of  computer-based  tasks  and  their  application  to  menu  selection  are 
also  discussed.  Chapters  3  through  6  present  a  series  of  empirical  studies 
that  test  hypotheses  about  perceptual  and  motor  aspects  of  menu  selection. 
They  focus  on  understanding  the  mechanisms  that  underlie  the  major 
findings.  Chapter  7  summarizes  the  results  and  discusses  the  implications 
of  the  research  for  both  theory  development  and  interface  design. 

Menus  Used  in  the  Experiments 

Many  different  styles  of  mouse-activated  menus  are  currently 
implemented  on  computer  systems  (Callahan  et  al.,  1988).  These  include 
pull-down,  click-open,  walking,  and  pie  menus  all  of  which  involve  very 
different  perceptual  features  and  action  sequences.  Little  empirical 
research  has  been  done  to  suggest  which  style  is  best.  One  possible 
research  approach  would  be  to  compare  performance  for  all  the  menu 
styles  in  one  set  of  tasks. 

However,  such  an  ambitious  undertaking  has  a  problem.  Because 
there  are  many  differences  in  the  physical  arrangements  and  actions 
associated  with  menu  styles,  it  would  be  difficult  to  determine  which 
attributes  are  responsible  for  any  performance  differences.  The  studies  in 
this  dissertation  have  reduced  the  potential  complexity  by  comparing  two 
styles  of  menus  that  are  perceptually  equivalent  but  require  different 
physical  actions  to  select  displayed  options  from  them.  To  be  specific,  we 
will  focus  on  click-open  and  walking  menus. 

Click-open  and  walking  menus  are  two  popular  styles  of  "pop-up" 
menu  used  in  advanced  graphical  workstations  (e.g.,  Sun,  Next).  Pop-up 
menus  are  invisible  when  a  person  enters  information  that  does  not  require 
the  menu;  yet  they  can  be  quickly  accessed  by  a  single  button  press.  When 
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the  button  of  a  mouse  is  depressed,  a  linear  list  of  menu  items  appears  or 
"pops  up"  at  the  location  of  the  cursor  on  the  screen. 

The  next  section  provides  detailed  descriptions  of  click-open  and 
walking  menus.  Figure  1.1  shows  the  physical  actions  required  for  a 
two-level  menu-selection  using  both  menu  styles. 

Comparison  of  Action  Sequences  For  Different  Menu  Styles 

Click-Open  Menus 

Click-open  menus  are  accessed  by  clicking  (pressing  and  releasing)  a 
mouse  button  on  a  computer  display.  Once  the  menu  is  visible,  the  cursor  is 
located  just  above  the  upper  left  comer  of  the  menu  (five  pixels  in  from  the 
edge)  and  then  moved  to  the  desired  menu  item.  A  command  is  selected 
with  a  click  of  the  mouse  button  anywhere  in  the  box  containing  the  menu 
item.  Upon  pressing  the  mouse  button,  the  next  menu  level  appears. 
Submenus  are  aligned  with  the  selected  higher-level  menu  item, 
overlapping  ten  pixels.  Once  the  mouse  button  is  clicked  in  the  final  menu, 
it  disappears  and  the  command  is  executed.  A  selection  can  be  aborted  by 
clicking  outside  the  menu  region. 

Two  defining  features  of  click-open  menus  are:  (1)  all  menu  items  are 
selected  with  a  click  of  the  mouse  button,  and  (2)  the  cursor  is  moved  with 
the  mouse  button  released  until  a  sequence  of  selections  is  completed  or  the 
button  is  clicked  outside  the  menu  region. 

Walking  Menu 

For  walking  menus,  a  menu  selection  is  initiated  by  pressing  and 
holding  the  mouse  button  down,  bringing  up  the  first  menu  level.  The 
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Click-open  menu  Walking  menu 


Figure  1.1  Physical  actions  reqxiired  by  menu  styles  for  two-level  selections. 


cursor  is  located  exactly  as  in  the  click-open  menu,  and  is  then  moved  to  the 
desired  menu  item.  A  command  is  selected  by  moving  the  cxirsor  into  a 
"hot  zone"  located  at  the  extreme  right  of  the  menu  item.  The  "hot  zone"  is 
a  rectangtilar  area  ten  pixels  wide,  running  from  the  top  to  the  bottom  of  the 
menu.  Perceptually,  the  "hot  zone"  is  identified  by  a  rightward  arrow  that 
is  aligned  with  the  left  edge  of  the  zone.  As  soon  as  the  cursor  crosses  into 
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this  region,  the  next  menu  level  appears,  aligned  with  the  selected  menu 
item,  overlapping  ten  pixels.  Selection  from  the  final  menu  is  made  by 
releasing  the  mouse  button  anywhere  in  the  selected  menu  item.  Upon 
release  of  the  button,  the  menu  disappears,  and  the  command  is  executed. 

A  selection  can  be  aborted  by  releasing  the  mouse  button  outside  the  menu 
region. 

Three  defining  features  of  walking  menus  are:  (1)  Higher-level 
menu  items  are  selected  by  moving  the  cursor  into  a  "hot  zone";  (2)  the 
cursor  is  moved  with  the  mouse  button  depressed;  (3)  final-level  menu 
items  are  selected  by  releasing  the  mouse  button. 

As  Figure  1.1  shows,  the  physical  actions  required  by  these  two  menu 
styles  are  quite  different.  Click-open  menus  require  a  click  (i.e.,  a  button 
press  and  release)  to  bring  up  the  first  menu,  and  an  additional  click  for 
selection  at  each  menu  level.  Walking  menus  require  a  single  press  (and 
hold)  to  bring  up  the  first  menu,  followed  by  a  single  button  release  at  the 
end,  regardless  of  the  number  of  menu  levels.  This  comparison  of  the 
number  of  button  presses  favors  the  walking  menu,  since  it  involves  fewer 
button  actions  than  are  reqmred  with  a  click-open  menu.  On  the  other 
hand,  a  walking  menu  imposes  some  severe  movement-path  constraints: 
The  horizontal  movement  from  the  first-level  menu  to  the  second  must  stay 
within  the  boimdaries  of  the  menu  box  or  else  the  wrong  second-level  menu 
will  be  opened.  In  contrast,  with  the  click-open  menu,  once  the  button  is 
clicked  to  bring  up  the  second-level  menu,  any  path  to  the  next  item  is 
allowed.  Consequently,  these  two  menu  styles  present  an  interesting  trade¬ 
off  between  the  minimization  of  button  presses  and  the  reduction  of  path 
constraints.  This  dissertation  assesses  the  relative  costs  of  these  design 
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features.  Also,  it  quantifies  the  impact  that  they  have  on  the  complexity  of 
planning  and  the  difficulty  of  executing  the  task  of  menu  selection. 


CHAPTER  2 

LITERATURE  REVIEW 


Introduction 

In  this  chapter,  I  cull  earlier  research  for  insights  about  the 
information-processing  demands,  research  paradigms,  and  models 
relevant  to  xmderstanding  menu  selection.  The  studies  reviewed  in  this 
chapter  come  from  two  quite  distinct  research  areas:  human  motor  control, 
and  human-computer  interaction.  Although  the  subject  matter  and 
research  approaches  of  these  disciplines  differ,  they  all  contribute  insight 
into  the  task  of  menu  selection.  This  review  is  organized  aroxmd  three 
major  topics:  (1)  the  control  of  aimed  movements,  (2)  visual  search  of 
menus,  and  (3)  performance  models  of  computer-based  tasks.  Research 
from  various  literatures  will  be  intermingled  as  they  pertain  to  these  topics. 

The  Control  of  Aimed  Movements 

Using  a  mouse  involves  a  strong  motor-skill  component  with 
coordination  of  highly  practiced  finger,  hand,  and  arm  movements.  One 
area  of  research  relevant  to  this  component  is  included  in  the  field  of 
human  motor  control.  Since  the  turn  of  the  century,  psychologists  and 
physiologists  have  studied  how  people  plan  and  control  movement 
(Woodworth,  1899).  The  findings  and  experimental  methods  developed  in 
these  studies  may  help  in  understanding  the  menu-selection  task  and  may 
provide  tools  to  explore  these  tasks  with  scientific  rigor.  In  particular,  the 
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literature  on  motor  control  includes  two  lines  of  research  relevant  to  the 
study  of  menu-selection;  (1)  studies  of  the  speed  and  accuracy  of  aimed 
movements,  and  (2)  studies  of  the  planning  and  execution  of  response 
sequences.  These  are  briefly  reviewed  here  with  special  emphasis  on  issues 
that  they  raise  about  the  menu-selection  task. 

Movement  Speed  and  Accuracy 

The  use  of  a  mouse  to  select  commands  from  a  computer  menu 
involves  speeded  movements.  The  user's  goal  is  to  position  a  display  cursor 
at  the  menu  item  by  moving  the  mouse.  Movements  like  this  have  been 
studied  extensively  in  the  field  of  human  motor  control  (e.g.,  Meyer  et  al., 
1988). 

The  oldest,  and  probably  best  understood  aspect  of  human  motor 
control  deals  with  the  relationship  between  the  speed  and  accuracy  of 
aimed  movements.  Woodworth  (1899)  was  one  of  the  early  researchers  to 
report  an  inverse  relationship  between  movement  speed  and  accuracy.  He 
found  that  as  movement  speed  increases,  accuracy  decreases. 

Another  major  breakthrough  in  understanding  the  relationship 
between  movement  speed  and  accuracy  came  with  a  formal  mathematical 
description  of  the  speed-accuracy  trade-ofiF  (Fitts,  1954).  Fitts  had  subjects 
move  a  stylus  back  and  forth  between  two  target  regions  as  quickly  as 
possible  while  keeping  their  error  rate  (target  misses)  low.  He  discovered 
that  the  average  movement  time  was  a  logarithmic  function  of  the  distance 
between  the  targets  divided  by  their  widths.  Known  as  Fitts’  law,  the 
function  is: 

Movement  Time  (M7^  =  a 6  logj  [2i)/W] 
where,  D  is  the  distance  between  the  centers  of  the  targets,  W  is  the  width  of 
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the  target  regions  toward  which  the  subject  moves,  and  a  and  b  are 
constants.  The  quantity  loga  [2D/W]  is  called  "the  index  of  difficulty"  (ID). 

Fitts'  law  is  a  good  description  of  movement  time  for  a  wide  variety  of 
situations.  For  example,  Fitts  and  Peterson  (1964)  demonstrated  that  it 
applies  when  a  subject  makes  a  discrete  movement  to  a  target.  Langolf, 
Chaffin  and  Foulke  (1976)  verified  Fitts'  law  for  tiny  finger  movements 
under  microscopic  conditions.  Fitts'  law  also  accounts  for  movement  times 
with  a  mouse  used  in  text  selection  (Card,  English,  &  Burr,  1978)  and  for 
graphical  editing  tasks  (Epps,  1986,  1987).  A  major  emphasis  of  the  present 
research  is  to  use  Fitts'  law  for  understanding  menu  selection. 

Device-Comparison  Studies 

One  reason  for  considering  the  mouse  as  a  selection  device  is  that 
many  empirical  studies  have  shown  it  to  be  a  good  one.  A  prevalent 
approach  to  research  on  selection  devices  in  the  field  of  human-computer 
interaction  is  the  comparative  study.  Here,  a  benchmark  movement  task 
(or  small  set  of  tasks)  is  selected,  and  several  selection  devices  are  used  to 
accomplish  the  task(s).  The  main  independent  variables  are  the  sizes  of  the 
movement  targets,  the  distances  to  them,  and  the  type  of  task.  The  main 
dependent  variables  are  movement  times,  error  rates,  and  user  preference 
ratings.  Typical  analyses  include  ANOVA's  for  determining  the  effects  of 
the  independent  variables,  calculation  of  the  learning  curve  for  each  device, 
and  a  regression  analysis  based  on  Fitts'  law  to  model  the  asymptotic 
performance  for  the  applicable  devices.  The  weight  of  evidence  indicates 
that  the  mouse  is  an  optimal  or  near  optimal  device  in  terms  of  both 
learning  and  performance  measures.  Three  illustrative  demonstrations  of 
this  include  an  early  study  by  Card,  English,  and  Burr  (1978)  and  two  recent 
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ones  by  Epps  (1986,  1987)  which  represent  the  best  published  work  to  date. 

Card,  English  and  Burr  (1978)  chose  text  editing  of  a  document  as 
their  benchmark  task.  Subjects  were  told  to  locate  a  word  on  a  display 
screen  and  to  select  that  word  by  positioning  a  cursor  over  it  as  quickly  as 
possible.  The  experiment  compared  performance  with  four  different 
selection  devices  (step  keys,  joystick,  mouse,  and  text  keys),  using  a  within- 
subjects  design.  Subjects  were  given  extensive  practice  with  each  device. 
Performance  with  the  mouse  exhibited  the  steepest  learning  curve,  and  the 
fastest  movements  once  learning  had  stabilized.  The  overall  speed 
advantage  of  the  mouse  was  accompanied  by  the  lowest  error  rate  (5%).  The 
highest  error  rate  was  found  for  step  keys  (13%).  Movement  times  for  the 
mouse  were  well  modeled  by  Fitts’  law: 

MT  =1030  +  96  ED,  r^  =  0.83  (time  in  msec).^ 

Epps  (1986)  chose  a  target-acquisition  task  as  his  benchmark.  Square 
targets  of  varying  sizes  and  screen  distances  were  selected  through  one  of 
six  devices  in  a  within-subjects  study.  The  devices  were  two  touchpads,  two 
joysticks,  mouse,  and  trackball.  Analyses  of  the  selection-time  data  showed 
that  the  mouse  and  track  ball  were  optimal  devices,  while  the  two  joysticks 
yielded  the  worst  performance.  Movement  times  for  each  of  the  devices 
were  well  modelled  by  Fitts'  law.  The  formula  for  the  mouse  was; 

MT  =  108  +  392  ID,  r^  =  0.70  (time  in  msec). 

In  a  follow-up  study,  Epps  (1987)  compared  performance  with  the 
same  six  selection  devices  in  a  set  of  seven  graphics  editing  tasks  (e.g.  line 
drawing,  object  selection,  object  resizing).  While  this  study  did  not 
manipulate  the  size  and  distance  of  the  targets  sufficiently  to  represent  the 

*For  the  calculation  of  ID,  Card  et  al.  used  Welford's  (1968)  correction.  For 
comparisons  sake,  all  ID  calculations  reported  in  this  dissertation  will  also  be  based  on 
Welford's  correction,  which  is  MT=  a  +  btDAV  +  .5). 


12 


times  with  Fitts'  law,  Epps  is  one  of  the  few  investigators  to  have  looked  at 
performance  in  more  than  one  benchmark  task.  His  data  indicated  that 
across  the  various  tasks,  subjects  performed  best  with  the  trackball  and 
mouse,  worst  with  the  two  joysticks.  The  trackball  and  mouse  were  also 
subjects'  favorite  devices,  while  the  absolute  touchpad  was  least  favored. 

Application  of  Fitts'  Law  to  Menu  Selection 

It  seems  plausible  that  the  task  of  menu  selection  with  a  mouse 
would  also  be  well  described  by  Fitts'  law.  If  so,  then  we  may  predict  menu- 
selection  times  and  guide  the  design  of  menus  to  speed  menu  access  and 
reduce  errors.  If  Fitts'  law  fits,  do  different  menu  styles  have  the  same  or 
different  parameter  values?  If  they  are  different,  then  knowing  the  values 
of  the  slope  and  intercept  for  various  menu  styles  will  allow  trade-off 
analyses  to  determine  which  is  best  tmder  different  conditions. 

A  pilot  study  with  walking  menus  has  yielded  some  evidence  that 
movement  times  for  the  menu- selection  task  accord  well  with  Fitts'  law 
(Walker,  Smelcer,  &  Nilsen,  1988).  The  time  to  select  the  first  item  in  a 
two-level  walking  menu  is  given  by  the  following  equation: 

MT=  303  +  185  ED,  r^  =  0.92  (time  in  msec). 

This  result  is  encouraging,  but  such  research  should  be  extended  to  a 
wider  variety  of  menu  styles  to  determine  how  robust  the  trade-off 
relationship  is. 

Special  Features  of  the  Menus 

Two  aspects  of  the  menus  examined  in  the  present  research  are  not 
ordinarily  studied  in  experiments  on  Fitts'  law:  (1)  movement  along  a 
constrained  path,  and  (2)  movement  with  constrained  finger/hand  postures. 
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In  most  experiments  on  Fitts'  law,  the  major  independent  variables 
are  the  target  width  and  distance.  The  movement  path  (i.e.,  the  trajectory) 
between  the  starting  point  of  the  movement  and  the  target  is  not  physically 
constrained,  but  is  assumed  to  be  a  straight  line.  This  latter  assumption 
appears  in  the  calculation  of  the  target  width  and  distance.  On  the  other 
hand,  walking  menus  involve  significant  path  constraints  and  curvilinear 
movements  in  going  from  one  menu  level  to  the  next.  We  want  to  determine 
whether  this  path  constraint  will  change  the  parameters  for  Fitts'  law  or 
invalidate  it  altogether? 

The  motor-control  literature  contains  only  one  set  of  studies  that 
examines  Fitts'  law  for  movements  with  path  constraints  of  the  t5T3e  that 
occur  in  walking  menus  (Kvalseth,  1973,  1975).  These  studies  involved 
moving  a  bail-point  pen  back  and  forth  between  two  targets  as  fast  as 
possible  while  trying  to  keep  the  movement  within  certain  spatial  bounds. 
Various  kinds  of  bounds  were  included.  The  one  most  like  that  of  the 
menu-selection  task  involved  movement  through  a  slot  of  various  heights 
situated  between  two  targets  (e.g.  ).  Based  on  his  results, 

Kvalseth  proposed  a  modification  of  Fitts'  law  that  adds  another  parameter. 
Kvalseth's  equation  was: 

MT=  -139.38  +  31.50  ID  +  25.83  [1/V],  ri  =  0.94  (time  in  msec). 

The  first  two  terms  of  this  equation  are  identical  to  those  in  Fitts'  law. 
The  "V"  here  is  the  height  of  the  center  slot.  Kvalseth  found  a  good  fit  for 
his  equation  in  the  slot  condition. 

Unfortunately,  he  did  not  report  the  fit  of  the  simpler  version  of  Fitts' 
law,  which  is  needed  to  see  how  much  improvement  was  made  by  adding 
another  parameter  to  it.  If  the  times  for  the  walking  menu  deviate  greatly 
from  Fitts'  law,  Kvalseth's  equation  may  be  a  good  candidate  for  a 
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replacement.  However,  if  Fitts’  law  provides  a  reasonable  fit  to  the  data,  it 
would  be  preferred  on  grounds  of  parsimony. 

A  second  salient  feature  of  the  present  menu  styles  is  that  they 
involve  constrained  versus  free  finger/hand  position.  This  feature  entails  a 
comparison  between  different  effector  systems  for  movement.  It  could  have 
a  significant  effect.  Langolf,  Chaffin,  and  Foulke  (1976)  compared  the 
parameters  of  Fitts'  law  for  movements  of  the  fingers,  the  wrist,  and  the 
whole  arm.  They  found  that  the  more  muscles  involved  in  the  movement, 
the  slower  the  movement  and  the  steeper  the  slope  in  Fitts’  law  (parameter 
b).  Their  results  would  imply  that  longer  selection  times  should  occur  for 
walking  menus,  because  they  require  keeping  the  button  depressed  during 
the  movement. 

A  recent  study  in  the  field  of  human-computer  interaction  addressed 
this  possibility  for  the  task  of  selecting  text  using  a  mouse.  Gillan  et  al. 
(1990)  examined  Fitts’  law  for  point-click  and  point-drag  sequences.  The 
point-click  sequence  consisted  of  moving  the  mouse  with  the  button 
released,  while  the  point-drag  sequence  involved  moving  the  mouse  with 
the  button  depressed.  Gillan  et  al.  (1990)  varied  the  distance  and  the  width 
of  the  targeted  text.  They  found  that  the  point-click  sequence  was 
significantly  faster  than  the  point-drag  sequence,  for  all  combinations  of 
distance  and  width.  The  point-click  sequence  was  influenced  by  the  width 
of  the  text,  while  the  point-drag  sequence  was  not.  Apparently,  for  the 
point-click  sequence,  the  effective  target  width  depended  on  the  entire  text 
target  while  for  the  point-drag  sequence,  it  depended  on  the  left  edge  of  the 
text.  Fitts’  law  accounted  quite  well  for  both  the  pointing  and  the  dragging 
segments  of  the  selections  (r^  ranging  from  0.88  to  0.99  for  the  best  fitting 
regressions). 
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Though  the  task  of  Gillan  et  al.'s  subjects  was  not  menu-selection, 
the  findings  from  it  do  have  implications  for  the  current  study.  They 
suggest  that  moving  with  a  mouse  button  depressed  is  more  complex  to 
plan  and  time  consuming  to  execute  than  moving  with  the  button  released. 
The  parameters  in  Fitts'  law  may  vary  with  the  physical  characteristics  of 
the  movement.  This  dissertation  examines  how  these  single  components 
combine  in  the  task  of  menu  selection. 

Planning  Response  Sequences 

This  dissertation  also  deals  with  the  phase  in  which  mouse 
movements  are  planned.  Research  done  on  the  planning  of  response 
sequences  can  provide  a  guide  toward  the  understanding  of  the  planning 
phase  of  menu  selection.  A  central  concept  in  motor  control  is  the  idea  of  a 
"motor  program".  A  motor  program  is  "a  set  of  muscle  commands, 
structured  before  a  movement  sequence  begins,  which  allows  the  entire 
sequence  to  be  carried  out  iminfluenced  by  peripheral  feedback"  (Keele, 
1968).  According  to  conventional  wisdom,  a  portion  of  the  planning  time  for 
a  movement  is  used  for  specifying  and  loading  the  muscle  commands  into 
the  motor  program,  which  then  automatically  controls  the  movement. 

Since  motor  programming  is  assumed  to  take  place  before  the 
movement  begins,  studies  of  is  focus  on  factors  that  influence  the  reaction¬ 
time  interval.  Two  kinds  of  reaction-time  (RT)  paradigms  have  been  used  to 
study  the  planning  of  movements:  simple  and  choice.  In  this  dissertation, 
the  user  always  has  advance  knowledge  of  the  item  to  be  selected;  so  only 
research  with  the  simple  RT  paradigm  is  discussed  here. 

In  simple  reaction  time  (SRT)  studies,  a  single  response  is  prepared 
on  each  trial,  and  execution  of  the  movement  is  signaled  by  a  reaction 
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signal.  This  allows  for  complete  and  unambiguous  planning  of  the 
movement.  The  SRT  represents  the  time  to  load  or  otherwise  prepare  an 
already  constructed  motor  program,  and  to  mobilize  the  muscles  to  begin 
the  movement.  Any  aspect  of  the  movement  that  lengthens  the  SRT  is 
attributable  to  the  effort  needed  to  prepare  a  motor  program.  Varying  the 
nature  of  the  required  movements  and  examining  the  effects  of  those 
changes  on  the  reaction  time  can  uncover  the  basic  unit  of  movement  in 
menu  selection. 

One  factor  that  affects  SRT  is  the  complexity  of  the  movement. 
Complexity  can  be  defined  as  the  number  of  goal-oriented  segments  in  a 
movement  (Henry  &  Rogers,  1960).  Research  on  various  classes  of 
movements  has  shown  that  SRT  increases  with  the  number  of  different 
segments  in  the  response.  For  example,  this  response  complexity  effect 
occurs  for  grasping  and  hitting  (Henry  &  Rogers,  1960),  ke3T5ressing  (Klapp, 
1977),  speech  and  typing  (Sternberg,  Knoll,  Monsell  &  Wright,  1978)  and 
tapping  movements  (Fischman,  1984). 

The  Fischman  study  is  most  relevant  to  menu  selection  with  a 
mouse.  In  it,  subjects  used  a  stylus  to  tap  a  series  of  metal  disks  (ranging 
from  one  to  five  targets)  arranged  either  in  a  straight  line,  or  in  a  staircase 
that  required  a  90°  change  of  direction  between  each  target.  The  movement 
trajectories  in  the  change-of-direction  condition  were  very  similar  to  those 
in  selection  from  multi-level  pop-up  menus.  The  number  of  different 
targets  (disks)  also  corresponded  nicely  to  the  nvunber  of  menu  levels  in  a 
hierarchical  menu. 

Fischman  found  that  SRT  increased  linearly  with  the  number  of 
target  disks,  for  both  the  straight  line  and  the  change-of-direction 
conditions.  This  complexity  effect  was  strongest  for  the  one  and  two  target 
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conditions.  Changes  in  movement  direction  did  not  affect  SRT.  The 
implication  here  is  that  adding  movement  segments  increases  the 
complexity  of  a  prepared  motor  program,  but  changing  direction  does  not. 

Replicating  Fischman's  findings  in  a  menu-selection  task  would 
have  important  implications  for  menu  design  (e.g.,  increasing  the  depth  of 
multiple  level  menus  can  only  be  achieved  at  the  cost  of  longer  planning 
time).  The  greatest  cost  may  occur  between  one  and  two  menu  levels,  with 
lesser  but  significant  effects  up  to  five  menu  levels.  However,  the  relative 
impact  of  the  complexity  effect  could  differ  for  menu-selection  tasks  and 
depend  on  menu  style.  If  the  change  of  direction  does  not  matter,  then  for 
experienced  users  who  know  the  location  of  the  menu  items  (and  hence  can 
prepare  a  complete  motor  program),  the  spatial  layout  of  the  submenus  will 
not  matter  in  planning  of  the  movement.  Of  course,  these  suggestions  must 
be  viewed  in  the  context  of  other  aspects  of  menu  selection,  such  as 
execution  time  and  visual  scanning. 

What  are  the  dimensions  of  complexity  that  influence  the  reaction 
time  for  mouse  movements?  Possible  candidates  include  mamber  of 
changes  of  direction,  distance  to  be  moved,  number  of  movement  segments, 
and  path  restrictions  during  the  movement.  Manipulating  the  depth  of  the 
menu  hierarchy  (number  of  menu  levels)  can  influence  extent,  changes  in 
direction,  and  number  of  movement  segments  in  the  response  sequence. 
The  comparison  of  click-open  and  walking  menus  will  provide  information 
about  the  influence  of  the  number  of  movement  segments  and  path 
restrictions  on  the  stage  of  response  programming. 

Visual  Search  Of  Menus 

Another  body  of  research  relevant  to  this  dissertation  deals  with  the 
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visual-search  process  in  menu  selection.  A  hallmark  study  by  Card  (1983) 
concluded  that  search  is  random  for  command  menus.  This  conclusion  is 
based  on  the  findings  that  (1)  there  are  no  differences  in  the  search  times 
for  items  at  different  positions  in  the  menu,  (2)  eye  movements  are  both 
downward  and  upward  instead  of  consistent  in  direction  and  (3)  the 
cumxilative  probability  of  locating  a  target  as  a  function  of  time  is  best  fit 
with  an  unsystematic  random-sampling  model. 

Other  researchers  (MacGregor,  Lee  &  Lam,  1986;  MacGregor  &  Lee, 
1987)  have  challenged  Card's  conclusion,  showing  that  his  empirical 
results  are  consistent  with  a  non-random  model  incorporating  redimdant, 
exhaustive,  serial  search.  Another  experiment  (Perlman,  1984)  produced  a 
serial-position  effect  for  menus  of  letters  and  numbers,  which  suggests  a 
serial  search  from  either  the  beginning  or  end  of  the  menu.  In  yet  another 
study  (Somberg,  Boggs  &  Picardi,  1982)  a  serial,  top-to-bottom,  self¬ 
terminating  search  was  found  for  categorical  information  in  a  menu.  The 
issue  of  random  versus  systematic  search  is  not  yet  settled. 

Nevertheless,  researchers  agree  that  the  nature  of  the  search  process 
changes  with  the  experience  of  the  user.  Novice  or  occasional  users  will 
take  a  long  time  to  search  menus.  With  increased  familiarity,  users  move 
towards  a  strategy  of  "direct  search"  where  they  fixate  immediately  on  the 
desired  menu  item  (Card,  1983;  MacGregor  &  Lee,  1987).  Once  the  user 
becomes  familiar  with  the  menus,  the  time  for  visual  search  is  greatly 
reduced  and  becomes  constant  in  the  predictive  model.  It  is  very  important 
to  understand  the  influence  that  visual  search  has  in  the  menu-selection 
task  for  users  with  different  knowledge  of  the  menu  contents. 

An  unanswered  question  concerns  when  visual  search  occurs  in  the 
process  of  menu-selection.  Many  of  the  above  studies  used  keypresses  to 
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select  information  from  a  menu.  When  a  selection  is  made  by  a  single 
keypress,  visual  search  may  be  completed  before  the  movement  is  initiated, 
since  information  from  the  visual  search  is  needed  in  order  to  know  which 
key  to  press.  However,  when  a  selection  is  made  using  a  mouse,  the 
movement  can  begin  before  the  visual  search  is  completed,  on  the  basis  of 
partial  information.  Alternatively,  one  can  move  the  cursor  while 
searching  for  the  correct  item  to  select. 

Previous  studies  have  assumed  that  visual  search  and  physical 
selection  are  independent  processes.  One  example  of  this  assumption 
comes  from  a  study  of  various  ways  to  deal  with  menu  items  that  are  only 
appropriate  in  certain  situations  (Francik  &  Kane,  1987).  In  the  authors' 
conclusion  they  state  that 

"the  results  ...  suggest  that  faster  selection  has  an  important 
visual  search  component.  Hence  deleting  inactive  items  from 
menus  should  help  users  select  items  more  rapidly,  regardless 
of  whether  the  selection  is  made  with  a  cursor  key,  a  letter,  or 
another  input  device  such  as  a  mouse." 

Their  assumption  that  the  visual-search  component  is  clearly 
separable  from  the  input  device  seems  tenuous  at  best.  The  physical  actions 
used  to  make  the  selection  could  influence  the  search  process.  For 
example,  typing  a  letter  may  not  change  the  display  until  the  menu  item  is 
selected,  and  it  provides  little  proprioceptive  feedback  until  the  key  is 
depressed.  In  contrast,  moving  a  mouse  changes  the  position  of  the  cursor 
on  the  display  and  provides  rich  proprioceptive  feedback  before  the  selection 
is  completed.  The  differences  in  visual  and  proprioceptive  feedback  for 
these  two  input  devices  could  result  in  very  different  search  strategies.  This 
dissertation  tests  this  possibility  by  exploring  whether  altering  the  physical 
actions  in  menu  selection  influences  the  search  process. 
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Performance  Models  Of  Computer-Based  Tasks 

Below  is  a  description  of  two  performance  models  in  the  field  of 
human-computer  interaction  that  can  be  applied  to  the  task  of  menu 
selection  and  that  make  predictions  about  users'  performance  in  menu- 
selection  tasks. 

Keystroke-Level  Model 

One  candidate  is  the  Keystroke-Level  Model  of  Card,  Moran,  and 
Newell  (1980,  1983).  It  predicts  the  amount  of  time  that  a  skilled  user  would 
take  in  performing  a  computer-based  task,  based  on  the  mental  and 
physical  operations  required  to  execute  the  task.  By  applying  a  series  of 
cognitive  and  motor  operators  to  each  of  the  steps  in  the  task,  the  model 
generates  a  total-time  prediction  for  that  task.  The  operators  in  the 
Keystroke-Level  Model  that  are  relevant  to  the  menu-selection  tasks  in  the 
present  study  are  listed  below; 

P  Pointing  to  a  target  on  the  screen  with  a  mouse,  1100  msec. 

k  Pressing  the  mouse  button,  200  msec. 

M  Preparing  mentally  for  a  physical  action,  1350  msec. 

The  time  parameters  for  these  operators  were  estimated  from 
empirical  studies  of  people  using  text  editors,  graphical  editors  and  some 
operating-system  commands.  By  using  a  set  of  heuristic  rules  to  determine 
which  operators  are  involved  in  a  task,  this  analysis  successfully  accounted 
for  over  90%  of  the  variance  in  the  total  task  times  (Card,  Moran,  &  Newell, 
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1983).  With  the  addition  of  a  perceptual-scanning  operator,  the  Keystroke- 
Level  Model  accurately  predicted  the  execution  time  for  a  set  of  spreadsheet 
tasks  (Olson  &  Nilsen,  1988). 

However,  a  number  of  questions  arise  when  applying  the  operators  of 
the  Keystroke-Level  Model  to  the  task  of  menu  selection  with  a  mouse. 

In  multi-level  menu  selections,  should  each  level  of  the  menu  be  considered 
a  target  on  the  screen  and  assigned  a  P  operator,  or  shoxild  the  entire  menu 
traversal  be  considered  as  a  single  pointing  operation?  Depending  on  the 
answer  to  this  question,  very  different  time  predictions  will  emerge.  For  a 
two-level  menu  selection  with  a  click-open  menu,  the  relevant  physical 
operators  include  two  button  presses  (at  200  msec  each)  and  either  one  or 
two  mouse  pointings  (at  1100  msec,  each)  yielding  a  time  estimate  ranging 
from  1.5  to  2.6  sec.  For  the  walking  menu,  the  relevant  physical  operators 
include  a  single  button  press,  and  either  one  or  two  mouse  pointing 
operators  for  a  range  of  1.3  to  2.4  sec  using  the  average  P  operator.^ 

Another  question  concerns  the  placement  of  the  M  operators.  Where 
in  the  process  of  menu  selection  does  mental  preparation  occur?  Is  mental 
preparation  required  for  traversing  each  level  of  a  multi-level  menu  (to 
retrieve  the  target  item  from  memory  or  acknowledge  that  the  correct  item 
has  been  reached),  or  does  a  single  mental  preparation  at  the  outset  of  a 
menu  selection  suffice  for  preparing  the  entire  movement?  Depending  on 
the  answer,  the  above  estimates  might  be  wrong,  because  they  assume  that 
all  mental  preparation  occurs  before  the  menu  is  accessed,  which  is  why 
the  M  parameter  does  not  appear  in  the  calculations. 

There  are  obviously  some  parameters  missing  from  the  Keystroke- 
Level  Model  that  may  be  relevant  to  selection  for  the  menu  styles  under 

*By  applying  Fitts'  law,  which  underlies  the  P  operator,  the  range  for  the 
click-open  menu  increases  from  1.2  to  3.4  sec,  and  for  the  walking  menu  1.0  to  3.2  sec. 
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study.  The  model  includes  no  perceptual  operator  to  represent  the  visual- 
search  process.  However,  in  a  recent  review  article  of  cognitive- 
engineering  models,  Olson  and  Olson  (1990)  recognized  this  shortcoming 
and  added  a  perceptual  operator  of  230  msec  for  a  saccadic  eye  movement 
and  an  operator  of  100  msec  for  simple  perception  and  recognition,  based  on 
previous  empirical  studies.  These  parameters  were  gleaned  from  research 
on  specialized  component  tasks  and  have  not  yet  been  verified  in 
combination  for  HCI-relevant  tasks. 

The  model  also  has  no  parameter  to  account  for  the  cost  of  path  constraints 
in  movement.  This  exclusion,  along  with  the  inclusion  of  the  button  press 
parameter  (k),  results  in  a  prediction  that  selection  from  walking  menus 
will  always  be  faster  than  selection  from  click-open  menus.  Futhermore, 
this  difference  should  increase  with  the  nrunber  of  menu  levels.  This 
increase  is  a  testable  prediction  that  will  reflect  on  the  efficacy  of  the 
Keystroke-Level  Model. 

A  more  basic  question  regarding  the  Keystroke-Level  Model  is 
whether  its  assumption  of  seriality  fits  the  task  of  mouse-based  menu 
selection.  The  model  assumes  that  the  total  selection  time  is  the  sum  of  a 
number  of  subcomponent  times.  This  assumption  may  be  valid,  but  another 
alternative  is  that  the  perceptual,  cognitive,  and  motor  processes  involved  in 
menu  selection  are  parallel.  For  example,  as  discussed  earlier  in  the 
section  on  visual  search  of  menus,  searching  for  a  command  and  moving 
the  cursor  down  the  menu  may  take  place  simultaneously.  The  exact 
relationship  between  the  two  processes  could  depend  on  the  knowledge  and 
experience  of  the  user.  When  the  user  does  not  know  the  location  of  the 
correct  command,  the  search  time  might  overshadow  the  motor  movement 
time.  However,  for  experienced  users  who  know  the  exact  identity  and 
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location  of  the  commands,  the  search  time  would  be  minimal,  and  the 
motor  time  would  determine  the  speed  of  performance.  Other  parallel 
processes  may  also  be  involved  in  menu  selection.  If  a  task  involves 
substantial  parallel  processes,  a  serial  model  would  overestimate  the  time 
to  accomplish  the  task. 

Critical-Path  Analysis 

One  approach  to  modelling  both  serial  and  parallel  processes  is 
critical-path  analysis,  a  technique  borrowed  from  operations  research 
called  (Malcolm,  Roseboom,  Clark,  &  Fazar;  1959).  This  technique  treats 
the  different  processes  used  to  accomplish  a  task  (perceptual,  cognitive,  and 
motor)  as  potentially  serial  or  parallel.  By  specifying  the  time  that  each 
process  takes  to  execute,  and  identifying  the  sequential  dependencies 
between  processes,  the  method  generates  a  total-time  estimate  for  the  task. 
The  total-time  estimate  is  reached  by  sxunming  the  duration  of  all  processes 
that  take  place  serially  and  only  the  longest  duration  of  any  processes 
executed  in  parallel.  The  shorter  processes  not  on  this  critical  path  are 
executed  for  free  as  long  as  some  other  process  does  not  delay  their  onset  or 
lengthen  their  duration.  This  framework  is  ideal  for  analyzing  the 
influence  of  a  users'  knowledge  and  experience  on  the  relationship  between 
searching  and  moving  in  a  menu. 

Several  psychological  studies  have  used  critical-path  analysis  in 
reaction-time  experiments.  It  has  been  applied  to  model  the  effect  of 
response  compatibility  in  the  Stroop  task  (Schweickert,  1978),  to  explain 
negative  interactions  between  figure-ground  contrast  and  response 
difficiilty  in  a  lexical-decision  task  (Hardzinski,  1980),  and  to  account  for  the 
pattern  of  results  from  various  stimulus-response  compatibility  tasks 
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(John,  1988). 

John  and  her  colleagues  have  successfully  used  critical-path 
analysis  to  model  the  overall  execution  time  for  a  variety  of  computer-based 
tasks  ranging  widely  in  complexity  and  length.  Tasks  to  which  she  has 
applied  her  model  include  transcription  typing  (John,  1988),  and  processing 
of  long  distance  calls  (Gray,  John,  &  Atwood,  1991). 

Not  enough  is  known  yet  about  the  task  of  menu-selection  to  propose  a 
fully  specified  critical-path  model  for  it.  If  the  serial  Keystroke-Level  Model 
proves  inadequate  to  predict  the  timing  of  menu  selection,  the  current 
research  will  reveal  some  of  the  sequential  dependencies  and  timing 
parameters  that  would  contribute  to  specifying  such  a  model. 

Summary 

The  present  review  of  the  literature(s)  has  generated  a  large  number 
of  questions,  and  several  specific  predictions  regarding  the  perceptual  and 
motor  factors  that  might  affect  the  task  of  menu  selection.  These  can  be 
summarized  in  a  small  set  of  central  questions.  The  next  chapters  will 
attempt  to  address  them  as  follows: 

(1)  Is  the  speed  and  accuracy  of  menu  selection  sensitive  to  changes  in 

the  physical  actions  required  by  different  menu  styles?  If  so, 
which  aspects  of  selection  are  affected  by  the  difference? 

(2)  How  are  the  applicability  and  parameter  values  of  Fitts'  law  affected 

by  the  subjects'  advance  knowledge  about  the  location  of  menu 
items? 
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(3)  Is  visual  search  random  or  systematic  in  mouse-based  menu 

selection? 

(4)  Are  visual  search  and  movement  serial,  independent  processes,  as 

assumed  in  the  literature  on  menu  selection? 

(5)  Is  menu  selection  with  a  mouse  best  characterized  by  a  serial  or  a 

parallel  model? 


CHAPTER  3 
EXPERIMENT  1 

Rationale 

This  experiment  was  designed  to  identify  the  principal  perceptual 
and  motor  factors  that  determine  performance  in  menu  selection  with 
click-open  and  walking  menus.  Regarding  motor  processes,  it  explores  the 
trade-off  between  multiple  button  presses  and  constraints  along  the  path  of 
movement.  The  effect  of  button  status  (pressed  or  released)  dvning  the 
major  movement  phase  is  also  explored.  These  two  motor  factors  are 
intentionally  left  confounded  in  this  initial  study  to  approximate  current 
menu  designs.  Finally,  the  applicability  of  Fitts’  law  to  these  styles  of  menu 
is  tested. 

The  perceptual  process  explored  here  is  visual  search.  We  examine 
how  visual  search  changes  with  different  levels  of  information  regarding 
the  location  of  menu  items.  The  current  experiment  includes  two  extremes 
of  this  continuum,  complete  advance  location  information,  and  no  advance 
location  information.  As  a  result,  it  may  be  determined  whether  visual 
search  is  best  described  as  an  random  or  systematic  process  under  these 
conditions. 

Finally,  Experiment  1  was  designed  to  examine  the  relationship 
between  the  search  and  motor  processes.  It  concerns  another  interesting 
question.  Are  they  independent  processes  as  previous  studies  have 
assumed,  or  do  the  actions  involved  in  menu  selection  and  the  process  of 
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searching  the  menus  interact? 


Method 

Subjects 

Twenty- three  right-handed  University  of  Michigan  students  served 
as  paid  subjects.^  They  had  no  apparent  motor  deficiencies.  All  of  them 
had  experience  using  a  mouse.  Each  subject  was  paid  $12  for  three,  1  hr 
sessions,  plus  bonuses  based  on  performance.^ 


Procedure 

The  procedure  involved  self-initiated,  speeded  selection  from  two-level 
menus.  At  the  beginning  of  each  trial,  subjects  were  shown  two  munbers 
that  identified  the  selection  targets  from  each  of  the  menu  levels  for  that 
trial.  When  they  had  memorized  the  numbers,  they  depressed  the  mouse 
button  to  display  the  first  level  of  the  menu,  and  then  they  made  their 
selections  as  fast  as  possible  while  minimizing  errors. 

The  sequence  of  events  during  a  trial  is  shown  in  Figure  3.1.  At  the 
beginning  of  each  trial,  subjects  placed  the  cursor  in  a  small  "go"  box  in  the 
upper  left-hand  corner  of  the  screen.  A  precue  appeared  directly  above  the 
box.  The  precue  consisted  of  the  menu  items  to  be  selected  on  each  menu 
level,  separated  by  commas.  Subjects  were  told  to  take  as  much  time  as  they 


twenty-four  subjects  were  run  in  the  study,  but  the  data  for  one  subject  were 
lost  because  of  a  computer  failure.  Of  the  remaining  subjects,  twelve  used  the  click-open 
menus,  and  eleven  used  the  walking  menus. 

*  The  bonuses  were  based  on  performance  and  calculated  for  each  block  of 
trials.  Money  was  awarded  to  the  subjects  based  on  the  average  selection  time  for  a 
block  of  trials.  One  cent  was  added  to  the  bonus  for  every  100  msec  under  four  seconds 
for  the  average  selection  time.  For  every  error  trial  in  a  block,  5  cents  were  deducted 
regardless  of  the  selection  time.  Thus,  an  average  correct  selection  time  of  2.5  sec  with 
no  errors  earned  15  cents,  while  the  same  time  with  two  errors  earned  5  cents.  Subjects 
were  told  that  the  bonus  could  not  take  away  from  their  base  pay  rate.  In  actuality,  all 
subjects  received  positive  bonuses  of  at  least  $5  over  the  course  of  the  experiment.  The 
bonus  system  was  explained  to  subjects  before  the  experiment  began. 
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needed  to  memorize  the  precues,  and  to  begin  the  menu  selection  when  they 
were  ready.  Upon  clicking  or  pressing  the  mouse  button  in  the  "go"  box,  the 


Screen  One 

When  cursor  is  moved  into  “go"  box 
the  precues  appear  above  the  box. 
Subjects  take  as  much  time  to 
memorize  the  precues  as  they  want. 


Screen  Two 

When  the  subject  is  ready  he 
clicks/presses  in  the  "go"  box.  The 
precues  disappear  and  the  menu  pops 
up.  He  then  moves  the  cursor  to  the 
first  menu  item  specified  in  the 
precue. 


3 ,4 


Screen  Three 

Upon  selecting  a  first-level  menu 
item,  the  second  level  menu  pops  up 
adjacent  to  the  first  menu  item.  The 
subject  moves  to  the  second  menu 
item. 


Screen  Four 

Upon  selecting  a  second-level  menu 
item  (or  selecting  a  region  outside  of 
the  menu),  the  menu  disappears  and 
the  trial  is  over.  A  box  containing 
accuracy  feedback  is  displayed  and  a 
new  ”go"  box  appears  in  the  top  right 
comer  of  the  screen. 
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Figure  3,1  Sequence  of  events  on  an  experimental  trial. 
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numbers  disappeared  and  the  subjects  made  their  menu  selections  as  fast 
as  possible.  Upon  completing  the  selection,  accuracy  feedback  was  flashed 
on  the  screen,  and  the  "go"  box  appeared  for  the  next  trial. 

Design 

The  twenty- three  subjects  were  divided  into  two  groups  (of  twelve  and 
eleven).  Each  group  made  menu  selections  from  a  single  style  of  pop-up 
menu  (click-open  or  walking).  All  subjects  selected  from  menus  having  two 
different  heights  (ten  pixels  and  twenty-five  pixels  per  item)  and  two 
different  organizations  (normal  and  random).  A  block  of  trials  always 
consisted  of  a  single  combination  of  menu  height  and  menu  organization. 
The  four  combinations  seen  by  each  subject  are  shown  in  Figure  3.2.  Each 
menu  level  consisted  of  the  digits  one  through  six  for  a  total  of  thirty-six 
distinct  menu  selections.  A  block  of  trials  consisted  of  two  practice  trials 
followed  by  one  of  each  of  the  thirty-six  menu  combinations  repeated  until 
they  were  selected  correctly.  At  the  end  of  a  block  of  trials,  subjects  were 
shown  the  average  total  selection  time  for  correct  trials,  the  number  of  error 
trials,  and  the  bonus  that  they  had  earned  based  on  their  performance. 

Each  subject  participated  in  three  sessions  held  on  consecutive  days. 
The  first  session  involved  training  the  subjects  on  the  menu  style  that  they 
would  use,  followed  by  two  blocks  of  trials  for  each  of  the  height  by 
organization  combinations.  None  of  the  data  from  the  first  day  are  included 
in  the  analysis  of  results.  The  first  day  served  strictly  as  a  familiarization 
and  practice  session.  The  second  and  third  sessions  were  for  data 
collection.  The  organization  of  menus  was  counterbalanced  across  days. 
Menu  height  was  alternated  within  a  session,  and  five  blocks  for  each 
menu  height  were  completed. 
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MENU 

ORGANIZATION 


MENU-ITEM  HEIGHT 
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Figure  3.2  Menu  height  by  menu  organization  combinations  for  Experiment  1. 

Results 

The  results  from  this  experiment  are  presented  in  four  sections:  (1) 
errors  in  menu  selection;  (2)  total  selection  time;  (3)  selection  times  for 
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various  phases  of  movement;  and  (4)  selection  times  based  on  the  serial 
position  of  the  first  menu  selection. 

Errors  in  Menu  Selection 

There  are  two  kinds  of  errors  possible  in  multi-level  menus.  Upon 
selecting  a  wrong  first-level  item  and  detecting  the  mistake,  it  is  possible  to 
recover  and  select  the  correct  first-level  item.  This  is  called  an  "inefficiency 
error".  Use  of  the  click-open  menus  produced  a  much  lower  rate  of 
inefficiency  errors  (0.71%)  compared  to  the  walking  menus  (6.2%)  [F(l,21)  = 
31.63,  p  <  .0001].  The  cost  in  time  for  these  is  also  quite  high:  Selections 
containing  an  inefficiency  error  took  1.1  sec  longer  on  average  than  did 
correct  selections. 

When  a  subject  selects  a  wrong  item  at  the  final  level  of  the  menu,  or 
fails  to  notice  an  incorrect  first-level  selection,  or  selects  a  region  outside  of 
the  menu,  then  the  menu  disappears.  This  precludes  recovery  and  is, 
therefore,  a  fatal  error.  The  rates  of  fatal  errors  were  very  low  for  both 
click-open  (1.6%)  and  walking  menus  (2.3%).  They  did  not  differ 
significantly  from  each  other  [F(l,21)  =  3.39,  p  >.05]. 

In  the  following  sections  for  the  time  results,  only  error-free  trials 
are  included. 

Total  Selection  Time 

Total  time  for  correct  menu  selections  is  operationally  defined  as 
starting  with  the  action  that  brings  the  menu  on  the  screen  (click  or  press) 
and  ending  with  the  action  that  completes  the  selection,  causing  the  menu 
to  disappear.  For  each  subject,  the  mean  selection  time  was  calculated  for 
each  combination  of  menu  organization,  menu  size,  and  block.  A  repeated 


32 


measures  analysis  of  variance  (ANOVA)  was  performed  on  the  mean 
selection  time  using  these  three  within- subject  variables  and  menu  style  as 
a  between-subject  variable. 

Each  menu-style  by  organization  by  menu-size  combination  was 
repeated  in  five  blocks  of  trials  during  the  data  collection  sessions  to  assure 
that  the  subjects  had  reached  asymptotic  performance.  ANOVA  results 
showed  that  there  was  a  block  effect  [F(4,84)  =  5.53,  p  <  .001].  The  range  of 
times  was  fairly  small  (1780  msec  to  1728  msec).  Subsequent  analyses 
indicated  that  the  block  effect  on  time  was  not  systematic;  it  remained  after 
eliminating  the  first  three  blocks  of  trials.  More  importantly,  eliminating 
early  blocks  did  not  change  the  results.  Subsequent  analyses,  therefore, 
include  all  blocks.  A  summary  of  the  actual  times  and  significance  levels 
for  the  variables  of  interest,  by  movement  phase,  is  shown  in  Table  3.1. 

Selection  times  were  significantly  faster  for  click-open  menus  (1680 
msec)  than  for  walking  menus  (1832  msec)  [F(l,21)  =  12.81,  p  <  .001]. 
Selecting  from  tall  menus  displayed  a  45  msec  speed  advantage  over 
selecting  from  short  menus  [F(l,21)  =  24.44,  p  <  .0001].  Not  surprisingly, 
menu  organization  had  the  largest  effect  on  the  selection  times,  with 
selections  from  normally-organized  menus  occurring  664  msec  faster  than 
those  from  randomly-organized  menus  [F(l,21)  =  1087,  p  <  .0001].  The  only 
significant  interaction  was  a  menu-style  by  organization  interaction; 
selecting  from  the  random  organization  was  especially  slow  for  the  walking 
menus  [F(l,21)  =  7.43,  p  <  .01].  The  effect  of  random  versus  normal 
organization  was  111  msec  less  for  click-open  menus  than  for  walking 
menus.  Figure  3.3  shows  the  menu-style  by  organization  interaction. 
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Click-open 

Walking 


Tall 

Short 


Table  3. 1  Selection  times  by  movement  phases  for  Experiment  1. 


MOVEMENT  PHASES 


Normal 

Random 


M 


by 

Menu 

Organization 


Click  Normal 
Click  Random 
Walk  Normal 
Walk  Random 


Total  Selection 
Time 

Start-Up 

First-Level 

Execution 

Second-Level 

Selection 

p<.001 

p  <  .01 

n.s. 

p  <  .0001 

1,680 

45 

781 

854 

1,832 

97 

752 

983 

p<.0001 

p  <  .05 

p  <  .01 

p  <  .0005 

1J30 

65 

760 

905 

1,775 

74 

774 

927 

p  <.0001 

p  <  .0001 

p<.0001 

p<.0001 

1,421 

33 

624 

764 

2,085 

106 

911 

1068 

p  <  .01 

n.s. 

n.s. 

p  <  .01 

1,374 

15 

642 

718 

1,986 

74 

921 

991 

1,471 

53 

604 

815 

2,193 

141 

899 

1152 

All  times  in  milliseconc 


Selection  Time  for  Movement  Phases 


The  total  selection  time  was  divided  into  three  movement  phases: 
start-up,  first-level  execution,  and  second-level  selection.  A  verbal 
description  of  these  movement  phases  is  given  at  the  beginning  of  each 
section.  A  more  detailed  operational  definition  of  the  movement  phases  is 
given  in  Appendix  B.  The  ANOVA's  for  these  phases  were  identical  in 


34 


form  to  the  one  for  the  total  selection  time.  The  three  significant  main 
effects  and  one  interaction  in  the  total  selection  times  will  be  examined  for 
each  movement  phase.  No  additional  interaction  effects  were  fovmd  to  be 
significant  for  any  of  the  movement  phases.  These  results  are  also 
summarized  in  Table  3.1 


MENU 

ORGANIZATION 

•  Normal 
n  Random 


Figure  3.3  Menu  style  by  organization  interaction  for  total  selection  time 
in  Experiment  1.  (Dashed  line  is  parallel  with  the  normal  organization  line) 


Start-Up  Phase 

The  start-up  phase  begins  following  the  action  to  bring  up  the  first- 
level  menu®  (button  click  for  click-open,  and  button  press  for  walking),  and 


®rhe  initial  button  press  to  display  the  first  menu  level  is  not  included  in  the 
analyses.  Instead,  the  event  timing  begins  when  the  menu  appears  on  the  screen. 
Likewise,  the  end  of  the  selection  is  defined  as  the  button  action  that  makes  the  menu 
disappear  (button  press  for  click-open,  button  release  for  walking  menus). 
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ends  when  intentional  movement®  of  the  mouse  begins.  This  is  not  a 
typical  reaction-time  measure  because  menu  selection  is  self-initiated, 
instead  of  being  a  response  to  some  externally  generated  signal.  It  is  best 
understood  as  representing  the  amoimt  of  programming  effort  required  to 
shift  from  the  initial  button  press  to  initiating  the  movement  of  the  entire 
mouse.  The  pattern  of  results  for  this  start-up  phase  mimics  the  total 
selection  times  (see  table  3.1). 

The  start-up  times  for  click-open  menus  were  much  faster  than  for 
walking  menus  [F(l,21)  =  8.99,  p  <  .01].  Both  the  times  for  click-open  menus 
(45  msec)  and  walking  menus  (97  msec)  are  considerably  faster  than  a 
typical  reaction  time. 

Tall  menus  had  slightly  faster  start-up  times  than  did  short  menus 
[F(l,21)  =  4.94,  p  <  .05].  The  magnitude  of  the  difference  is  9  msecs,  which  is 
13%  of  the  overall  mean  start-up  time  of  70  msec 

Once  again,  menu  organization  had  the  largest  impact  on  the 
duration  of  the  start-up  phase,  with  start-up  times  being  73  msec,  faster  for 
normally-organized  menus  than  for  randomly-organized  menus  [F(l,21)  = 
28.27, p  <.0001]. 

The  menu-style  by  organization  interaction  failed  to  reach 
significance  for  the  start-up  phase  [F(l,21)  =  1.11,  p  >.30].  The  mean  times 
ranged  from  only  15  msec  for  beginning  to  move  in  normally-organized 
click-open  menus  to  141  msec  for  randomly-organized  walking  menus. 
High  between- subject  variability  in  the  start-up  times  for  the  random! y- 


®rhe  data  files  for  this  experiment  include  an  entry  only  when  the  mouse 
moves.  Occasionally  the  first  entry  is  very  fast  and  followed  by  a  significant  pause 
before  the  next  entry.  This  is  caused  by  muscle  tremor,  or  unintentional  mouse 
movement  which  accompanies  the  pressing  of  the  mouse  button.  To  overcome  this,  the 
movement  start  is  operationally  defined  as  the  first  of  three  consecutive  recordings  in 
the  data  file  each  of  which  have  a  duration  of  less  than  40  msec.  This  ensures  that  the 
movement  is  sustained  and  intentional. 
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organized  menus  suggests  that  individuals  adopted  different  strategies, 
with  some  subjects  beginning  to  move  immediately,  and  others  visually 
searching  before  beginning  to  move  the  mouse. 

First-Level  Execution  Phase 

This  movement  phase  commences  when  the  subject  begins 
intentional  movement  of  the  mouse,  and  ends  when  the  first  menu  item  is 
selected,  bringing  up  the  second  menu  level.  For  the  click-open  menu,  the 
termination  point  is  a  button  press  anywhere  in  the  target  menu  item;  for 
the  walking  menu,  it  is  signalled  by  moving  the  cursor  into  the  "hot  zone" 
region  at  the  right  of  the  menu  item.  (Refer  to  Figure  1.1  for  a  visual 
comparison  of  these  actions.) 

The  first-level  execution  is  the  only  movement  phase  for  which  no 
difference  occurred  between  click-open  and  walking  menus  [F(l,21)  =  1.28,  p 
>.25].  In  fact,  the  trend  is  in  the  opposite  direction  to  that  of  the  other 
movement  phases. 

The  first-level  execution  was  14  msec  faster  for  tall  menus  than  for 
short  menus  [F(l,21)  =  7.82,  p  <  .01].  This  result  is  interesting  because  Fitts’ 
law  predicts  that  the  times  should  be  the  same,  since  the  ID  values  were 
identical.’ 

The  organization  effect  is  highly  significant  in  the  expected  direction 
[F(l,21)  =  262,  p  <  .0001].  Selecting  from  normal  menus  was  287  msec  faster 
than  selecting  from  random  menus. 

The  menu-style  by  organization  interaction  is  not  evident  in  the 

’One  interesting  feature  of  many  menus  is  that  the  distance  of  the  menu  items 
and  their  size  are  perfectly  correlated.  If  the  menu  items  are  of  equal  size  in  a  linear 
list,  as  they  are  in  the  menus  under  study,  the  ID  is  equivalent  regardless  of  the  height 
of  the  items.  Knowing  the  serial  position  of  the  item  is  all  that  is  needed  to  calculate  the 
ID.  The  ID  values  for  this  experiment  range  from  0  for  the  first  menu  item  to  2.59  for 
the  sixth  menu  item. 


first-level  execution  phase  [F(l,21)  <  1].  As  in  the  start-up  phase,  the  pattern 
is  similar  to  the  one  foimd  for  the  overall  selection  time. 

Second-Level  Selection  Phase 

This  movement  phase  begins  with  the  appearance  of  the  second-level 
menu,  and  ends  with  the  action  that  completes  the  menu-selection  (button 
press  for  click-open,  and  button  release  for  walking).  It  is  the  longest,  and 
the  most  complex  movement  phase  in  terms  of  information-processing 
demands.  Here  subjects  must  verify  that  the  correct  first-level  menu  item 
was  selected,  search  for  the  second-level  menu  item,  move  to  it,  verify  that 
the  entire  menu  selection  is  complete,  and  finally  execute  the  button  action 
that  will  finish  the  trial. 

The  pattern  of  results  for  the  second-level  menu  selection  duplicates 
the  total  selection  time  in  both  significance  levels  and  magnitude  of  the 
effects.  Selecting  from  click-open  menus  was  129  msec  faster  than  selecting 
from  walking  menus  [F(l,21)  =  69.83,  p  <  .0001].  Selecting  from  tall  menus 
was  22  msec  faster  than  selecting  from  short  menus  [F(l,21)  =  12.03,  p  < 

.005].  Selecting  from  normally-organized  menus  was  304  msec  faster  than 
selecting  from  randomly-organized  menus  [F(l,21)  =  773,  p  <  .0001].  The 
menu  style  by  organization  interaction  is  also  evident,  with  walking  menus 
being  more  affected  by  the  random  organization  (160  versus  97  msec)  than 
were  click-open  menus  [F(l,21)  =  8.57,  p  <  .01]. 

Serial-Position  Functions 

Examining  the  selection  times  in  terms  of  the  serial  position  of  the 
menu  items  is  required  to  evaluate  both  the  visual  search  and  Fitts’  law  for 
the  motor  component  of  menu  selection.  The  start-up  and  first-level 
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execution  phases  are  combined  in  the  analyses  of  the  serial-position 
functions  for  several  reasons.  It  is  necessary  that  the  tests  concerning 
visual  search  and  Fitts'  law  be  based  on  the  same  data  in  order  to  uncover 
any  perceptual-motor  interactions.  For  visual  search,  the  shape  of  the 
serial-position  function  indicates  whether  the  search  process  is  systematic 
(selection  time  increases  with  serial  position)  or  unsystematic  (  no 
difference  in  selection  time  with  serial  position).  The  index  of  difficulty  for 
Fitts'  law  is  determined  by  the  size  and  distance  of  the  targets,  which  is 
correlated  with  the  serial  position  of  the  menu  items. 

Comparing  the  serial  position  functions  for  normally  and  randomly- 
organized  menus  also  reveals  the  relationship  between  visual  search  and 
movement.  A  comparison  of  the  shapes  of  the  functions  reveals  how 
positional  uncertainty  in  the  randomly-organized  menus  affects  visual 
search  and  movement.  As  mentioned  earlier,  high  between- subject 
variability  for  the  start-up  times  suggests  that  some  individuals  started  to 
search  the  menu  before  moving  the  mouse  and  others  started  to  search 
after.  Thus,  to  capture  the  entire  search  process,  both  the  start-up  and 
first-level  execution  phases  are  examined. 

The  first-level  execution  phase  comes  closest  to  being  a  discrete 
movement  to  a  bounded  target  with  advance  information  of  the  target 
location.  Adding  the  start-up  phase  increases  the  intercept  and  has 
minimal  effect  on  the  slope  of  Fitts'  law.  Since  the  slope  parameter  reflects 
the  motor  complexity  of  the  movement,  the  start-up  phase  is  included  for 
the  present  test  of  Fitts'  law. 

The  second-level  selection  phase  is  more  complicated.  As  noted 
earlier,  this  phase  has  many  information-processing  tasks  embedded  in  it. 
Also,  the  beginning  point  of  this  phase  is  very  different  for  the  click-open 
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and  walking  menus.  At  the  appearance  of  the  second-level  in  the  click-open 
menu,  the  cursor  is  typically  located  outside  the  second-level  region  and 
stationary.  For  the  walking  menu,  the  cursor  is  already  in  the  second-level 
region  and  moving  when  the  second-level  appears.  Thus,  the  purest 
examination  of  the  perceptual  and  motor  interactions  involve  analyzing  the 
combined  start-up  and  first-level  menu  execution  times. 

Figure  3.4  shows  the  serial-position  functions  for  the  four 
combinations  of  menu  style  and  menu  organization.  Here  the  functions  are 
grouped  according  to  menu  organization.  Not  surprisingly,  the  selection 
times  are  always  slower  for  the  random  menu  than  for  the  normal  menu 
items.  The  shapes  of  the  functions  are  quite  different  for  the  normal  and 
random  menus.  Each  must  be  examined  separately  to  determine  the 
regression  function  that  best  characterizes  the  pattern  of  selection  times. 

The  data  for  the  normally-organized  menus  are  well  characterized  by 
Fitts'  law  regardless  of  menu  style.®  The  regression  lines  and  equations 
are  shown  in  Figure  3.5. 

The  slope  is  slightly  shallower  for  the  click-open  menus  MT  =  395.7  + 
116.7(ID),  r^=.98  (time  in  msec),  than  for  the  walking  menus  MT  =  361.6  -h 
148.3(ID),  r^=.96  (time  in  msec).  These  two  fimctions  suggest  that  for 
normally-organized  menus,  the  selection  time  is  dominated  by  motor 
processes,  with  direct  visual  search  adding  a  constant  increment. 


®For  the  calculation  of  the  index  of  difficulty  (ID),  the  straight  line  vertical 
distance  is  measured.  The  width  is  calculated  as  the  vertical  height  of  the  menu  item. 
ID's  were  calculated  in  the  same  manner  for  both  walking  and  click-open  menus  even 
though  the  target  for  the  walking  menus  could  be  considered  to  be  the  "hot  zone"  at  the 
right  edge  of  the  menu  item  (see  Figure  1.1),  suggesting  a  distance  measure  of  the 
diagonal  distance  and  a  width  calculation  along  the  axis  of  primary  movement.  The 
rationale  for  using  the  simpler  vertical  calculations  of  ID  for  both  menu  styles  is 
two-fold.  First,  from  a  software  designer's  perspective,  the  calculation  of  vertical  ID  is 
much  simpler.  Secondly,  it  really  doesn’t  make  a  great  deal  of  difference  in  the 
relative  values  of  the  ID's,  whether  the  vertical  or  the  diagonal  distance  is  used  so  the 
parsimonious  vertical  calculation  is  preferred.  If  Fitts'  law  does  not  hold  for  the 
walking  menus,  the  diagonal  measure  of  distance  will  be  used. 
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Figure  3.4  Serial-position  functions  for  Experiment  1  separated 
by  menu  style  and  menu  organization. 


The  best  ftmction  for  the  randomly-organized  menus  is  linear  with 
serial  position  as  the  regressor  (Figure  3.6).  Excluding  the  first  menu  item 
results  in  a  very  good  fit  for  both  the  click-open  MT  =  534.2  +  118.3(menu 
item),  r^=.99  (time  in  msec)  and  walking  [MT  =  571.4  4-  131.4(menu  item), 
r^=.99  (time  in  msec)  menus.  This  suggests  that  a  systematic,  top-to- 
bottom,  visual  search  governs  the  selection  time  for  the  randomly-organized 
menus.  Apparently  the  motor  processes  involved  in  moving  the  mouse  are 
not  affecting  the  selection  time,  since  the  curve  obtained  for  the  normally- 
organized  menus  does  not  hold  for  the  random  menus. 

The  selection  time  for  the  first  item  of  random  menus  is  longer  than 
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Figure  3.5  Fitts'  law  for  normally  organized  menus  in  experiment  1 
separated  by  menu  style. 


predicted  by  the  search  function.  One  plausible  explanation  for  this  is  that 
the  visual  search  and  movement  are  initiated  in  parallel.  Since  the  visual 
search  takes  longer  than  the  mouse  movement  in  random  menus,  the 
subject  will  frequently  have  moved  the  cursor  past  the  first  menu  item  by 
when  he  has  identified  it  as  the  target  item.  This  results  in  a  corrective 
measure,  yielding  a  longer  time  to  select  the  first  menu  item. 

Discussion 

The  results  of  Experiment  1  clearly  show  that  the  current 
understanding  of  mouse-based  selection  from  hierarchical  menus  is 
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incomplete  and  flawed  in  some  respects. 


Figure  3.6  Linear  functions  for  the  randomly  organized  menus 
in  experiment  1  separated  by  menu  style.  (Excluding  the  first 
menu  item) 


First,  selections  from  the  click-open  menu  are  faster  than  from  the 
walking  menu;  this  disagrees  with  the  prediction  of  the  Keystroke-Level 
Model  outlined  in  Chapter  2.  Our  data,  thus  far,  indicate  that  making 
selections  by  moving  through  a  spatially-constrained  path  with  the  mouse 
button  depressed  is  slower  and  more  error-prone  than  making  selections  by 
moving  through  unconstrained  paths  and  clicking  at  the  beginning  (or  end) 
of  each  movement  segment.  Current  models  quantify  the  effect  of  the 
number  of  button  presses  and  imconstrained  movement  to  bounded  targets. 
No  existing  model  addresses  either  the  button  status  while  moving,  or 
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spatial  constraints  along  the  path  of  movement.  However,  both  of  these 
warrant  further  exploration,  based  on  the  results  of  this  experiment. 

The  present  subjects  selected  items  slightly  faster  from  tall  than  from 
short  menus,  contrary  Fitts'  law.  Since  target  size  and  distance  are 
perfectly  correlated  in  linear  menus  such  as  those  of  Experiment  1,  Fitts’ 
law  predicts  identical  selection  times  for  all  menu  heights.  In  Experiment 
1,  however,  subjects  paid  more  attention  to  target  size  than  target  distance. 
This  may  have  happened  because  the  menu  items  were  embedded  in  a  set  of 
similar  targets,  not  isolated  as  in  many  Fitts  studies.  The  most  salient 
common  feature  of  the  targets  was  the  size  of  the  menu  items,  which  might 
bias  the  subjects'  performance. 

The  effects  of  the  independent  variables  in  this  experiment  do  not 
seem  to  be  clearly  localized  in  any  one  phase  of  the  menu-selection  process. 
Rather,  they  seem  to  occur  in  every  phase  of  the  movement  to  var5dng 
extents.  The  largest  effects  appear  in  the  second  menu-selection  phase, 
which  has  the  greatest  number  of  information  processes,  including 
evaluation  of  first-level  selection,  planning,  search,  and  execution  of  the 
second-level  selection  followed  by  evaluation  of  the  entire  menu-selection. 

The  organization  of  the  menus  proved  to  be  the  strongest  determinant 
of  selection  time.  This  factor  also  revealed  the  most  about  the  process  of 
visual  search  in  menu  selection.  It  also  helped  manifest  the  relationship 
between  search  and  motor  processes  used  in  the  selection. 

The  significant  interaction  between  menu  style  and  menu 
organization  gives  evidence  for  the  non-independence  of  the  search  and 
motor  processes.  According  to  Sternberg's  (1969a)  additive-factor  logic, 
factors  that  affect  different  processes  in  a  serial-stage  model  should  have 
additive  effects  on  reaction  times  for  discrete  responses.  Generalizing  this 
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logic  to  selection  times,  the  interaction  in  Experiment  1  suggests  one  of  two 
possibilities:  (1)  visual  search  and  motor  processes  in  the  menu-selection 
task  occur  during  a  common  stage  of  processing,  or  (2)  a  serial  model  is  not 
an  adequate  characterization  for  the  task  of  menu  selection. 

The  serial-position  functions  (Figure  3.4)  support  the  second  of  these 
possibilities.  The  linear  functions  for  the  selection  from  the  random  menus 
is  well  explained  by  a  systematic,  serial-search  model  with  seemingly  little 
or  no  influence  of  the  motor  processes  except  for  a  small  constant  time 
added  in  selecting  from  the  walking  menus.  In  contrast  the  log-linear 
functions  for  the  normal  menus,  which  obey  Fitts'  law,  suggest  a  fast  direct 
search  that  adds  a  constant  to  the  selection  time.  The  divergence  of  these 
fiinctions  is  evidence  for  the  non-seriality  of  perceptual  and  motor 
processes. 

A  serial  model  predicts  that  the  shape  of  the  serial  position  curve  for 
the  normally-organized  menus  would  be  reflected  in  the  curve  for  the 
randomly-organized  menus.  The  time  for  the  visual  search  required  by  the 
randomly-organized  menus  would  add  to  the  motor  time  for  the  selection. 
The  shape  of  the  curve  for  the  randomly-organized  menus  depends  on  the 
natime  of  this  search  process.  Given  a  random  search  process,  foimd  by 
Card  (1983),  the  curves  for  the  random  menus  would  have  the  identical 
shape  as  the  curves  for  the  normal  menus.  The  visual  search  adds  a 
constant  to  the  selection  time,  regardless  of  the  serial  position  of  the  menu 
item.  Given  a  systematic  search  process,  the  serial  position  function  for  the 
random  menus  would  be  steeper  than  the  fimction  for  the  normal  menus, 
but  still  curved.  A  serial  model  that  would  account  for  the  present  data 
would  have  to  posit  an  exponentially  growing  search  function  which  is  an 
exact  mirror  image  of  Fitts’  law  found  for  the  normally-organized  menus. 
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Such  a  function  is  implausible  for  visual  search.  The  alternative 
explanation  of  concurrent  motor  and  search  processes  is  preferred. 

Experiment  1  has  revealed  new  insights  into  the  nature  of  visual 
search  and  the  control  of  aimed  movements  in  mouse-based  menu 
selection.  We  have  obtained  evidence  of  search  and  movement  functions 
that  are  highly  dependent  on  the  amount  of  advance  location  information 
available  to  the  user.  This  outcome  counters  the  claim  that  the  time  for 
visual  search  and  selection  are  independent  of  the  motor  processes  involved 
in  the  menu  selection.  The  empirical  evidence  refutes  one  example  of  a 
serial  stage  model  of  menu  selection  (Keystroke-Level  Model)  and  points 
toward  a  the  need  for  a  process  model  with  both  serial  and  concurrent 
processes  (Critical-Path  Model).  The  subsequent  experiments  in  this 
dissertation  will  explore  these  conclusions  in  greater  detail. 


CHAPTER  4 
EXPERIMENT  2 


Rationale 

The  data  from  Experiment  1  showed  that  click-open  menus  were 
superior  to  walking  menus  in  terms  of  movement  speed  and  errors. 
However,  some  questions  remain  imanswered. 

One  question  concerns  what  is  the  underlying  motor  mechanism  for 
the  speed  difference.  The  two-level  menus  in  Experiment  1  varied  on  two 
motor  factors:  the  trade-off  between  the  number  of  button  presses  and  path 
constraint,®  and  the  status  of  the  mouse  button  during  movement.  The 
resialts  of  Experiment  1  did  not  reveal  which  of  these  factors  play  a  role  in 
the  selection-time  difference  since  both  of  them  are  varied  there. 

Experiment  2  addresses  this  question  by  var3ring  only  button  status 
during  movement  while  eliminating  the  path-constraint  factor.  This  is 
accomplished  by  using  single-level  menus.  With  only  a  single  level,  there  is 
no  path  constraint  for  either  click-open  or  walking  menus,  the  size  and 
location  of  the  movement  targets  are  identical  (the  entire  menu  item),  and 
the  number  of  button  actions  is  equal  (one  action  to  display  the  menu,  and 
one  to  select  the  menu  item).  The  only  difference  between  the  menu  styles 
involves  the  mouse  being  moved  with  the  button  either  released  (click-open) 


®rhis  is  a  tradeoff  because  the  two  menu  styles  minimize  one  of  these  factors  at 
the  expense  of  the  other.  Walking  menus  minimizes  the  number  of  button  actions 
required  for  selection  (one  press  and  one  release  regardless  of  the  number  of  menus 
levels),  while  adding  a  path  constraint  for  moving  between  menu  levels.  Click-open 
menus  eliminate  all  path  constraints  at  the  cost  of  requiring  extra  button  presses  for 
each  menu  level. 
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or  depressed  (walking). 

If  the  difference  between  click-open  and  walking  menus  is  replicated 
in  Experiment  2  it  will  support  the  hypothesis  that  holding  the  mouse 
button  down  slows  menu  selection.  If  no  difference  occurs,  it  will  suggest 
that  the  trade-off  between  button  presses  and  path  constraint  is  a  primary 
contributor  and  should  be  the  focus  of  further  work. 

A  second  question  raised  by  Experiment  1  concerns  the  nature  of  the 
visual  search  that  occurs  in  mouse-activated  menu  selection.  The  data 
suggest  that  this  search  is  best  modeled  as  a  systematic,  top-to-bottom, 
process.  What  remains  unanswered  is  whether  the  search  is  self 
terminating,  exhaustive,  or  some  combination  of  the  two. 

This  question  will  be  addressed  by  varying  the  number  of  items  on  the 
menu.  With  a  strictly  self-terminating  search,  the  number  of  items  in  a 
menu  should  not  affect  the  serial-position  fimction.  An  identical, 
monotonically  increasing  function  should  be  found  for  each  menu  size.  In 
contrast,  a  strictly  exhaustive  search  would  yield  a  flat  serial-position 
function  with  the  y-intercept  increasing  as  the  number  of  items  on  a  menu 
increases.  A  search  process  combining  self- terminating  and  exhaustive 
components  would  predict  a  monotonically  increasing  serial-position 
function,  where  additional  menu  items  increase  the  selection  time  for  each 
location  on  the  serial-position  function. 

Varying  the  number  of  items  in  the  menu  may  also  give  insight  into 
the  nature  of  the  motor  processes  imderl3dng  menu  selection.  For  example, 
it  increases  the  range  of  ID  values  available  to  evaluate  the  robustness  of 
Fitts'  law  for  the  normally-organized  menus.  Also,  the  serial-position 
curves  for  the  normally-organized  menus  may  reveal  if  increasing  the  total 
number  of  movement  targets  on  the  screen  slows  the  movement. 
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Method 

Subjects 

Eight  right-handed  University  of  Michigan  students  served  as  paid 
subjects.  They  had  no  apparent  motor  deficiencies.  All  of  them  were 
experienced  mouse  users.  Each  subject  was  paid  $15  for  two,  1  1/2  hour 
sessions,  plus  bonuses  based  on  performance. 

Procedure 

The  procedure  involved  self-initiated,  speeded  selection  from  single- 
level  menus.  At  the  beginning  of  each  trial,  subjects  were  shown  a  digit 
that  identified  the  target  item  for  that  trial.  When  they  had  memorized  the 
digit,  they  depressed  or  clicked  the  mouse  button  to  display  the  menu,  and 
then  made  their  selection  as  fast  as  possible  while  minimizing  errors. 

Design 

Each  of  the  eight  subjects  used  both  styles  of  pop-up  menu  (click-open 
and  walking).  Menu  style  was  varied  across  days.  All  subjects  selected 
from  menus  having  different  lengths  (three,  six,  or  nine  items)  and 
different  nximeric  organizations  (normal  or  random).  Each  block  of  trials 
consisted  of  a  fixed  combination  of  menu  style,  menu  length,  and 
organization.  The  six  combinations  seen  by  each  subject  are  shown  in 
Figure  4.1.  A  block  of  trials  consisted  of  two  practice  trials  followed  by  three 
with  each  of  the  menu  items  repeated  until  each  was  selected  correctly. 
Each  combination  of  menu  length  by  numeric  organization  was  presented 
in  six  blocks  during  the  course  of  the  experiment. 
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Figure  4.1  Menu  organization  and  menu  length  combinations 
used  in  experiment  2. 


Results 

The  results  of  Experiment  2  are  presented  in  four  sections;  (1)  errors 
in  menu-selection;  (2)  total  selection  time;  (3)  selection  times  for  various 
phases  of  movement;  and  (4)  serial-position  fimctions  for  menu  selection. 
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Errors  in  Menu  Selection 

When  selecting  from  single-level  menus,  every  error  is  fatal,  whether 
it  involves  selecting  a  wrong  menu  item  or  stopping  outside  the  entire 
menu  region.  The  error  rates  were  low  for  both  click-open  (4.77%)  and 
walking  menus  (4.16%);  they  did  not  differ  significantly  [F(l,7)  <1].  As  in 
the  first  experiment,  only  error-free  trials  are  included  in  the  following 
analyses. 

Total  Selection  Time 

Total  selection  time  is  operationally  defined  as  starting  with  the 
action  that  first  displays  the  menu  on  the  screen,  and  ending  with  the 
action  that  completes  the  selection  to  make  the  menu  disappear.  For  each 
subject,  the  mean  selection  time  was  calculated  for  each  combination  of 
menu  style,  menu  organization,  menu  length,  and  block.  An  ANOVA  was 
performed  on  the  mean  selection  time  as  a  fimction  of  these  four  within- 
subject  variables. 

Each  combination  of  menu  style,  organization,  and  length  was 
repeated  in  six  blocks  of  trials  to  assure  that  subjects  reached  asymptotic 
performance.  There  was  a  significant  block  effect  [F(5,35)  =  5.01,  p  <  .001]. 
Removing  the  first  block  of  trials  from  the  analysis  eliminated  the  block 
effect  [F(4,28)  <  1].  Therefore,  all  analyses  are  performed  on  blocks  two 
through  six  to  examine  asymptotic  performance.  A  summary  of  the  actual 
times  and  attained  significance  levels  for  the  variables  of  interest,  separated 
by  movement  phase,  is  shown  in  Table  4.1. 

Selecting  from  cli.ck-open  menus  was  faster  than  selecting  from 
walking  menus  [F(l,7)  =  7.26,  p  <  .05].  The  average  selection  time  for  click- 
open  menus  was  743  msec  compared  to  819  msec  for  the  walking  menus,  a 
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Table  4. 1  Selection  times  by  movement  phases  for  Experiment  2. 


MOVEMENT  PHA 

SES 

EFFECTS 

Total  Selection 
Time 

Start-Up 

First-Level 

Execution 

p  <  .05 

n.s* 

n.s. 

Click-open 

743 

61 

682 

Walking 

819 

90 

730 

Menu 

Length 

p<.0001 

n.s. 

p  <  .0001 

Three 

567 

66 

501 

Six 

788 

72 

717 

Nine 

988 

88 

900 

Menu 

Organization 

p  <  .0001 

p<.01 

p  <  .0001 

Normal 

580 

26 

554 

Random 

982 

124 

858 

Menu  Length 

p  <  .0001 

p  <  .05 

p<.0001 

Menu  Stvle 
by 

Menu 

Organization 

n.s. 

n.s. 

n.s. 

All  times  in  milliseconds 


difference  of  76  msec  (almost  exactly  half  of  the  menu-style  effect  for  two- 
level  menus  in  Experiment  1).  Selecting  menu  items  was  slower  with  the 
button  depressed  than  with  the  button  released.  Selection  times  were  402 
msec  faster  with  normal  menus  than  with  random  menus  [F(l,7)  =  185.18,  p 
<  .0001].  Menu  length  also  had  a  strong  effect  on  selection  times  [F(2,14)  = 
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165.42,  p  <  .0001].  This  is  not  surprising,  since  the  average  movement 
distance  increases  with  menu  length.  There  was  a  significant  interaction 
between  menu  organization  and  length  in  the  data  [F(2,14)  =  55.61,  p  <  .0001]. 
As  menu  length  increased,  the  difference  in  selection  time  between  normal 
and  random  menus  increased  as  well.  Also,  the  interaction  between  block 
and  menu  organization  was  borderline  in  significance  [F(4,28)  =  2.64, 
p=.055].  Normal  menu-selection  times  exhibited  a  small  but  consistent 
decrease  over  the  course  of  the  experiment  (from  611  to  553  msec),  while 
random  menu-selections  varied  unsystematically  across  blocks.  The 
interaction  between  menu  style  and  organization  found  in  Experiment  1  did 
not  occur  here  [F(1,7)<1].  There  was  no  evidence  that  the  button  status 
while  moving  the  mouse  differentially  influenced  the  visual  search  in 
single-level  menus.  All  other  interactions  failed  to  reach  significance. 

Selection  Time  for  Movement  Phases 

The  total  selection  time  was  divided  into  two  movement  phases:  the 
start-up  phase  and  first-level  execution  phase.  A  verbal  description  of  these 
movement  phases  is  given  at  the  beginning  of  each  section.  A  more  detailed 
operational  definition  of  the  movement  phases  is  given  in  Appendix  B. 
ANOVA's  for  these  phases  were  identical  to  those  for  the  total  selection 
time. 

Start-Up  Phase 

As  in  Experiment  1,  the  start-up  phase  began  after  the  action  to 
display  the  first-level  menu  (button  click  for  click-open,  and  button  press  for 
walking),  and  it  ended  when  intentional  movement  of  the  mouse  began. 

Only  the  main  effect  of  menu  organization  reached  significance  for 
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the  start-up  phase  [F(l,7)  =  12.67,  p  <  .01].  Both  the  menu-style  and  menu- 
length  effects  were  in  the  expected  direction  but  not  significant.  The  start¬ 
up  time  for  click-open  menus  was  61  msec  compared  to  90  msec  for  walking 
menus  [F(l,7)  =  3.97,  p  >  .05].  For  the  menu-length  effect  the  average  start¬ 
up  times  for  the  3,  6,  and  9  item  menus  were  66,  72,  and  88  msec  respectively 
[F(2,14)  =  3.58,  p  >  .05].  The  only  significant  interaction  involved  menu 
organization  by  length  [F(2,14)  =  4.53,  p  <  .05].  It  paralleled  the  results  on 
total  selection  time;  the  difference  between  normal  and  random  menus 
increased  with  the  longer  menus. 

First-Level  Execution  Phase 

This  movement  phase  starts  when  the  subject  begins  intentional 
movement  of  the  mouse  and  ends  when  the  menu  item  is  selected.  The 
termination  event  is  a  button  press  anywhere  in  the  menu  item  for  the 
click-open  menus,  and  a  button  release  anywhere  in  the  menu  item  for  the 
walking  menus.  The  results  for  this  movement  phase  were  identical  to 
those  foimd  for  the  total  selection  time,  except  that  no  significant  menu- 
style  effect  occurred. 

The  difference  between  selecting  from  click-open  (682  msec)  and 
walking  menus  (730  msec)  was  not  statistically  significant  for  the  first-level 
execution  phase  [F(l,7)  =  4.21,  >  .05].  Selecting  from  normal  menus  was 

304  msec  faster  than  selecting  from  random  menus  [F(l,7)  =  176.85,  p  < 
.0001].  Menu  length  also  had  a  significant  effect  on  selection  times  [F(2,14)  = 
155.62,  p  <  .0001].  There  was  a  significant  interaction  between  menu 
organization  and  length  [F(2,14)  =  32.67,  p  <  .0001].  As  menu  length 
increased,  the  difference  in  selection  time  for  normal  versus  random 
menus  increased  as  well.  There  was  one  additional  significant  interaction 


between  menu-organization  and  block  [F(4,28)  =  2,96,  p  <  .05],  Normal 
menu-selections  showed  a  small  but  consistent  improvement  over  the 
course  of  the  experiment,  (from  568  to  533  msec),  while  the  time  for  selecting 
from  random  menus  varied  unsystematically  across  blocks. 

Serial-Position  Functions 

As  in  Experiment  1,  the  cleanest  examination  of  the  perceptual  and 
motor  interactions  involve  analyzing  the  serial-position  functions  for 
combined  start-up  and  first-level  menu-selection  times. 

Figure  4.2  shows  the  serial-position  functions  for  the  four 
combinations  of  menu  style  and  menu  organization  in  Experiment  2.  As  in 
Experiment  1  with  two-level  menus,  the  selection  time  was  always  slower 
for  the  random  menu  items  than  for  the  normal  menu  items.  The  shapes 
of  the  functions  are  also  very  similar  to  those  from  Experiment  1. 
Additionally,  the  magnitude  of  the  effects  are  similar  across  experiments, 
as  shown  in  Figure  4.3. 

The  data  for  the  normally-organized  menus  are  well  characterized  by 
Fitts'  law.  With  click-open  menus  MT  =  289.6  +  177.1(ID),  r^=.98  (time  in 
msec);  with  the  walking  menus  MT  =  328.9  +  206.8(ID),  r^=.99  (time  in 
msec).  These  functions  suggest  that  the  selection  time  with  normal  menus 
is  dominated  by  motor  processes.  Both  the  slope  and  the  intercept  are  lower 
for  the  click-open  menus  than  for  the  walking  menus,  which  reinforces  the 
conclusion  that  selecting  from  the  walking  menus  is  more  complex 
motorically. 

The  data  for  the  random  menus  appear  to  be  a  linear  function  of 
serial  position.  If  the  first  menu  item  is  excluded  from  the  data,  linear 
fimctions  fit  the  remaining  results  well  for  both  the  click-open  MT  =  515.8  + 


116.7(menu  item),  r^=.99  (time  in  msec)  and  walking  menus  MT  =  597.2  + 
123.6(menu  item),  r^=.99  (time  in  msec).  These  fimctions  suggest  that  a 
systematic,  top-to-bottom,  visual  search  dominated  the  selection  time  for  the 
random  menus. 

The  serial-position  fimctions  for  each  menu  length  collapsed  across 
menu  style  (Figure  4.4)  reveal  more  about  the  visual  search  in  menu 
selection. 

Looking  first  at  the  random  menus,  there  is  a  clear  increase  in 
selection  time  with  more  menu  items.  The  first  three  serial  positions  show 
a  steady  increase  in  selection  time  with  menu  length.  This  indicates  an 
exhaustive  component  of  the  search.  The  monotonically  increasing 
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Figure  4.4  Serial-position  functions  separated  by  menu  length 
for  Experiment  2. 
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functions  for  all  three  menu  lengths  indicates  a  self-terminating 
component  of  the  search  as  well.  The  evidence  points  to  a  visual  search 
combining  exhaustive  and  self-terminating  processes. 

The  serial-position  curves  for  the  normal  menus  are  qualitatively 
different.  Here  the  menu  length  has  little  or  no  impact  on  the  selection 
time.  This  agrees  with  the  suggestion  from  previous  research  on  the  visual 
search  of  menus  that  a  single  saccadic  eye  movement  to  the  menu  item  is 
possible  when  the  user  knows  the  location  of  the  menu  item  in  advance. 

Another  feature  of  the  serial-position  curves  for  normal  menus  is 
that  they  have  the  same  shape  regardless  of  menu  length.  This  means  that 
Fitts'  law  applies  equally  well  to  menus  of  differing  lengths. 


Discussion 

Interpretation  of  Results  from  Experiment  2 

The  results  of  Experiment  2  suggest  that  the  difference  in  selection 
time  between  click-open  and  walking  menu  depends  on  the  status  of  the 
mouse  button  during  the  selection  movement.  However,  the  present 
analysis  of  the  movement  phases  does  not  isolate  this  effect  in  a  single 
phase.  In  general,  the  pattern  of  results  for  each  movement  phase  was 
similar  to  the  pattern  for  total  selection  time,  though  the  effect  was 
generally  weaker  during  the  movement  phases.  Notably,  menu  style  did  not 
affect  either  movement  phase,  but  the  trend  favored  the  click-open  over  the 
walking  menu.  These  borderline  results  may  stem  from  the  high  degree  of 
between-subject  variability  in  the  duration  of  the  movement  phases.  This 
variability  may  be  caused  by  subjects  adopting  different  strategies 
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concerning  when  they  commence  visual  search  and  when  they  begin 
moving  the  mouse. 

Examination  of  the  serial  position  functions  separated  by  menu 
length,  indicates  that  the  search  process  for  random  menus  has  an 
exhaustive  component.  A  strictly  self-terminating  search  cannot  accovmt 
for  the  observed  menu  length  effect.  The  lack  of  a  menu  length  effect  for  the 
normal  menus  supports  the  finding  from  previous  research  that  the  search 
process  is  minimized  with  advance  location  information. 

Comparison  of  Results  for  First  and  Second  Experiments 

The  difference  in  error  rates  between  the  menu  styles,  which  was 
present  in  the  first  experiment  with  two-level  menus  was  absent  in  the 
second  experiment  with  one-level  menus.  This  suggests  that  the  additional 
errors  on  walking  menus  in  Experiment  1  were  caused  primarily  by  the 
path  constraint  on  moving  from  the  first  to  the  second  level.  This  constraint 
was  eliminated  in  Experiment  2  and  the  differences  in  error  rates 
disappeared  as  well. 

Another  result  foimd  in  the  first  but  not  second  experiment  is  the 
interaction  between  menu  style  and  menu  organization.  For  single-level 
selections,  walking  menus  were  not  more  affected  by  the  random 
organization  of  menu  items  than  were  click-open  menus.  This  suggests 
that  the  interaction  between  search  and  movement  is  related  to  the  added 
complexity  of  moving  along  a  constrained  path  (from  level  to  level)  rather 
than  being  related  to  the  button  status  during  the  movement. 

Turning  to  some  commonalities,  the  serial-position  functions  were 
qualitatively  and  quantitatively  similar  across  the  two  experiments. 
Selection  times  for  randomly-organized  menus  were  best  fit  by  a  linear 
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fvinction  in  which  each  menu  item  was  scanned  in  approximately  125 
msec.  Selection  times  for  normally-organized  menus  were  well 
characterized  by  Fitts'  law  in  the  range  of  three  through  nine  menu  items. 
Moreover,  the  parameters  of  all  the  fimctions  did  not  depend  on  either  the 
total  number  of  menu  levels  in  the  selection  or  on  whether  menu  style  was 
a  between  or  within-subjects  variable.  This  is  reassuring  as  it 
demonstrates  the  robustness  of  the  findings  across  experimental 
conditions. 


CHAPTERS 
EXPERIMENT  3 

Rationale 

The  resiilts  from  Experiment  2  show  that  one  important  factor  in  the 
speed  of  mouse-based  menu  selection  is  the  state  of  the  mouse  button.  From 
a  software  designer's  perspective,  the  study  might  well  end  here  with  a 
conclusion  that  clicking  is  faster  than  dragging  for  menu  selection  as  well 
as  for  other  computer-based  tasks  (Gillan  et  ah,  1990).  However,  from  a 
psychological  perspective,  there  is  still  one  important  question  to  address. 
What  is  the  mechanism  responsible  for  this  difference? 

Experiment  3  was  designed  to  test  two  alternative  hypotheses  about 
what  causes  the  observed  difference  between  moving  with  the  mouse  button 
depressed  and  moving  with  the  mouse  button  released.  One  possibility  is 
that  the  difference  might  be  caused  by  some  central-processing  demands. 
Alternatively,  it  might  be  caused  by  some  piu*ely  peripheral,  physical  aspect 
of  the  movements.  These  hypotheses  may  be  expressed  in  more  detail  as 
follows: 

1)  The  difference  for  dragging  versus  clicking  stems  from  a  greater 
load  on  working  memory.  This  is  because  one  must  remember  when  to 
keep  the  button  depressed  and  when  to  release  it  in  walking  menus,  but  not 
in  click-open  menus. 
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2)  The  difference  stems  from  increased  friction  between  the  mouse 
and  the  pad  caused  by  the  continuous  pressure  on  the  depressed  button. 

The  task  chosen  to  manipulate  the  load  on  working  memory  involved 
memory  search  of  short  lists  (Sternberg,  1969b).  In  the  standard  version  of 
this  task,  the  subject  memorizes  a  set  of  letters  or  numbers,  is  shown  a 
probe  letter  (or  number)  that  may  or  may  not  be  a  member  of  the  memorized 
set,  and  then  has  to  determine  as  quickly  as  possible  if  the  target  belongs  to 
the  set,  and  respond  accordingly.  For  this  task,  numerous  studies  have 
shown  that  reaction  time  increases  linearly  with  the  set  size.  The 
empirical  evidence  suggests  that  the  search  through  working  memory  is 
serial  and  exhaustive.  The  memory-search  task  has  been  used  in  many 
studies  as  a  measure  of  mental  workload  (Wickens,  1986). 

For  present  purposes,  the  memory-search  task  was  modified  slightly 
to  fit  with  our  menu-selection  task.  Two-level  menus  were  included  here. 
The  first-level  menus  were  numeric,  and  identical  to  those  used  in 
Experiment  1.  The  second-level  menus  were  alphabetic,  and  each  menu 
item  was  identified  by  a  different  letter.  All  selections  were  made  from 
randomly-organized  menus  in  order  to  preclude  users  from  associating 
numbers  with  spatial  menu  positions.  The  precues  for  the  numbers  were 
the  same  as  the  menu  item  to  be  selected,  just  as  in  the  previous 
experiments.  However,  the  precues  for  the  letters  consisted  of  one,  two,  or 
four  letters  that  constituted  a  memory  set.  One  and  only  one  of  the  letters 
appeared  on  the  second-level  menu,  constituting  a  memory  probe.  The 
subjects  had  to  select  the  correct  first-level  menu  item,  and  then  find  the 
letter  in  the  second-level  menu  that  had  been  a  member  of  the  memory  set. 
All  trials  had  a  member  of  the  memory  set  in  the  second-level  menu. 
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Examples  of  the  menus  and  precues  in  this  experiment  are  shown  in 
Figure  5.1. 
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Figure  5.1  Examples  of  menus  with  different  memory-set 
sizes  used  in  Experiment  3. 


One  expected  result  is  that  selection  time  should  increase  with  the 
size  of  the  memory  set.  Additionally,  if  selecting  from  the  walking  menus 
does  involve  extra  working  memory  capacity,  and  if  the  memory-search 
task  sufficiently  occupies  that  capacity,  then  an  interaction  between  menu 
style  and  set  size  shordd  occur.  Increasing  set  size  should  degrade 
performance  more  for  walking  menus  than  for  dick-open  menus. 

In  order  to  vary  a  peripheral  factor  separate  from  memory  load, 
friction  between  the  mouse  and  the  surface  on  which  the  mouse  moved  was 
varied.  This  was  accomplished  by  having  half  the  menu  selections 
performed  on  the  cloth  side  of  a  mouse  pad,  and  the  other  half  on  the 
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rubberized  backing  of  the  pad;  the  rubber  backing  of  the  mouse  pad  had  a 
much  greater  level  of  fHction  than  did  the  cloth  side. 

If  friction  is  a  major  determinant  of  movement  speed,  then  selection 
times  should  be  significantly  slower  on  the  high-friction  surface. 
Additionally,  if  pressing  the  mouse  button  raises  the  friction  level  more  for 
the  walking  menus  than  for  the  click-open  menus,  an  interaction  between 
menu  style  and  friction  level  should  occur.  With  menu  style,  memory  set 
size,  and  friction  level  as  the  main  independent  variables.  Experiment  3 
was  run  to  test  these  two  hypotheses. 

Method 


Subjects 

Twelve  right-handed  University  of  Michigan  students  served  as  paid 
subjects.  The  subjects  had  no  apparent  motor  deficiencies.  All  of  them 
were  experienced  mouse  users.  Each  subject  was  paid  $10  for  a  two  hour 
session,  plus  bonuses  based  on  performance. 

Procedure 

The  procedure  involved  self-initiated,  speeded  selection  from  two-level 
menus.  At  the  beginning  of  each  trial,  subjects  were  shown  precues 
consisting  of  a  single  digit  for  the  first-level  selection,  and  a  set  of  one,  two, 
or  four  letters  as  a  precue  for  the  second-level  selection.  When  they  had 
memorized  the  precues,  subjects  depressed  the  mouse  button  to  display  the 
first-level  menu,  and  proceeded  to  make  their  two  selections  as  fast  as 
possible  while  minimizing  errors. 
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Design 

The  twelve  subjects  were  divided  into  two  groups.  Each  group  made 
menu  selections  at  a  single  friction  level  (high  or  low).  Subjects  selected 
from  both  styles  of  menu  (click-open  and  walking),  and  they  received  all 
memory-set  sizes  (one,  two,  and  four).  All  menus  were  randomly  organized 
for  each  trial.  A  block  of  trials  always  consisted  of  a  single  combination  of 
menu  style  and  memory-set  size.  First-level  targets  were  identified  by 
single  digits  1-6,  and  second-level  targets  were  identified  by  single  letters, 
requiring  a  total  of  36  distinct  two-level  menu  selections.  A  block  of  trials 
consisted  of  two  practice  trials  followed  by  each  of  the  36  menu 
combinations,  each  of  which  was  repeated  xmtil  a  correct  selection  from  it 
occurred. 

The  first  half  of  the  experiment  involved  selecting  from  either  click- 
open  or  walking  menus;  following  a  break,  subjects  selected  from  the  other 
menu  style  for  the  remainder  of  the  session.  At  the  beginning  of  each  half, 
subjects  were  instructed  about  the  particxalar  menu  style  for  that  half,  and 
they  were  given  one  practice  block  to  familiarize  themselves  with  it. 
Following  practice,  each  memory-set  size  was  presented  in  three  blocks  of 
trials. 


Results 

The  results  are  presented  in  three  sections:  (1)  errors  in  menu 
selection;  (2)  total  selection  time;  and  (3)  selection  time  for  movement 
phases. 

Errors  in  Menu  Selection 

As  in  Experiment  1,  recoverable  efficiency  errors  and  non- 
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recoverable  fatal  errors  were  possible  in  this  two-level  menu  selection. 
Inefficiency  errors  occurred  more  often  with  walking  (1.8%)  than  with 
click-open  (0.5%)  menus  [F(l,10)=32.9,  p  <  .001].  Fatal  errors  were  more 
frequent  than  inefficiency  errors.  The  average  fatal-error  rate  was 
significantly  greater  for  click-open  menus  (3.1%)  than  for  walking  menus 
(2.3%)  [F(l, 10)= 13.88,  p  <  .005].  Only  error-free  trials  are  used  in  the 
following  time  analyses. 

Total  Selection  Time 

There  were  three  experimental  factors  in  this  study:  menu  style, 
memory-set  size,  and  friction  level.  The  actual  times  and  attained 
significance  levels  for  these  factors  are  shown  in  Table  5.1. 

Selecting  from  the  click-open  menus  (2,248  msec)  was  169  msec  faster 
than  selecting  from  the  walking  menus  (2,417  msec)  [F(l,10)=11.09,  p  <  .01]. 
The  size  of  the  memory  set  had  a  significant  effect  on  selection  time 
[F(2,20)=5.209,  p  <  .05],  with  the  average  selection  time  increasing  linearly 
with  the  size  of  the  memory  set.  Surprisingly,  friction  level  did  not  affect  the 
speed  of  selection  [F(l,10)  <  .01].  One  possible  explanation  for  this  is  that  by 
chance,  subjects  in  the  high-friction  condition  were  more  skilled  than  those 
in  the  low-friction  condition.  Another  possibility  is  that  subjects 
compensated  for  the  higher  friction  without  much  cost  or  difficulty.  The 
only  significant  interaction  occurred  between  menu  style  and  block 
[F(2,20)=5.3,  p  <.05].  The  selection  times  for  the  click-open  menu  exhibited  a 
small  but  consistent  improvement  from  block  one  to  block  three.  The 
walking-menu  selections  were  fastest  for  the  second  block  and  slower  for 
the  final  block  of  trials. 
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Table  5.1  Selection  times  by  movement  phases  for  Experiment  3. 


MOVEMENT  PHASES 

EFFECTS 

Total  Selection 
Time 

Start-Up 

First-Level 

Execution 

Second-Level 

Selection 

p  <  .01 

p  <  .005 

n.s. 

p  <  .05 

Click-open 

2,248 

98 

942 

1208 

Walking 

2417 

173 

977 

1267 

n.s. 

n.s. 

n.s. 

n.s. 

Low 

2,341 

158 

987 

1197 

High 

2,324 

113 

933 

1278 

Memorv 
Set  Size 

p  <  .05 

n.s. 

n.s. 

p  <  .05 

1 

2,138 

130 

951 

1057 

2 

2,323 

136 

974 

1212 

4 

2,537 

141 

954 

1443 

Menu.StYk 

by 

Block 

Click  .Walk 

p  <  .05 

lUS. 

n.s. 

p  <  .05 

1 

1 

2,271  2,478 

1,225  1,273 

2 

2 

2,260  2,371 

1,227  1,243 

3 

3 

2,213  2,403 

1,173  1,285 

All  times  in  milliseconds 

Two  critical  interactions  that  would  have  differentiated  between  the 


alternative  hypotheses  were  not  present  here.  The  interaction  between 
menu  style  and  memory-set  size,  which  would  have  supported  the  central 
memory-load  hypothesis,  was  not  significant  [F(2,20)=1.6,  p  >,2].  The 
interaction  between  menu  style  and  fnction,  which  would  have  supported 
the  peripheral  motor  hypothesis,  was  also  absent  [F(2,20)<1]. 
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Selection  Time  for  Movement  Phases 

The  total  selection  time  was  divided  into  three  phases:  start-up, 
first-level  execution,  and  second-level  selection.  The  operational  definitions 
for  the  movement  phases  were  the  same  as  those  for  Experiment  1  (see 
Appendix  B  for  details).  The  ANOVA's  for  these  phases  were  identical  in 
form  to  that  used  for  the  total  selection  time.  Only  the  significant  results 
are  presented  below. 

Start’Up  Phase 

The  only  significant  effect  in  the  start-up  phase  came  from  menu 
style  [F(l,10)  =  12.67, p  <  .01];  the  average  start-up  time  for  click-open  menus 
was  98  msec  compared  to  173  msec  for  walking  menus. 

First-Level  Execution  Phase 

None  of  the  factors  in  the  experiment  affected  this  phase  of  movement 
significantly,  although  the  absolute  time  comparisons  favor  the  click-open 
menu  (942  msec)  over  the  walking  menu  (977  msec). 

Second-Level  Selection  Phase 

In  this  experiment,  selecting  from  the  second-level  menu  was  clearly 
the  most  complex  phase.  The  subjects  had  to  search  the  randomly 
organized  letters  in  the  menu,  and  then  match  them  with  the  letters  that 
they  had  memorized  at  the  beginning  of  the  trial. 

The  pattern  of  results  for  the  second-level  selection  phase  is  very 
similar  to  the  total  selection  time.  Selecting  from  click-open  menus  (1208 
msec)  was  faster  than  selecting  from  walking  menus  (1267  msec)  [F(l,10)  = 
5.04,  p  <  .05].  The  size  of  the  memory  set  also  affected  the  selection  time 
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[F(2,20)  =  5.05,  p  <  .05],  The  friction  level  did  not  significantly  affect  the  time 
[F(1,10)<1].  Finally,  the  interaction  between  menu  style  and  block  found  in 
the  total  selection  time  is  also  evident  in  this  phase  of  movement  [F(2,20) 
=4.22,  p  <  .05]. 

Discussion 

Most  results  of  Experiment  3  are  not  surprising.  Selecting  from 
click-open  menus  was  faster  than  selecting  from  walking  menus,  thus 
confirming  the  results  from  the  previous  two  experiments.  The  magnitude 
of  the  difference  is  similar  for  simple  two-level  numeric  menus  (152  msec) 
and  two-level  menus  involving  a  memory  search  (169  msec).  The  motor 
differences  between  click-open  and  walking  menus  are  not  overwhelmed  by 
adding  a  modest  amovmt  of  cognitive  complexity  to  the  menu  selection.  In 
terms  of  critical-path  analysis,  the  button-depression  effect  remains  on  the 
critical  path  when  a  memory  search  is  added.  The  addition  of  the  memory- 
search  task  did  affect  the  second-level  menu  selection  as  expected.  No 
evidence  was  fotmd  for  the  added  complexity  influencing  earlier  phases  of 
menu  selection. 

It  is  puzzling  that  friction  level  did  not  affect  either  selection  times  or 
error  rates.  Because  of  the  small  sample  size  and  between-subject  design,  it 
is  tempting  to  attribute  this  result  to  the  vagaries  of  random  assignment.  If 
the  same  nxill  effect  was  fovuid  in  a  within-subject  design,  or  with  a  much 
larger  sample  size,  it  would  suggest  that  people  can  easily  adjust  their 
muscular  force  to  accommodate  externally  imposed  difficulties. 

Unfortimately,  this  experiment  did  not  reveal  the  mechanism  for  the 
slower  selection  times  associated  with  button  depression.  Adding  both 
central  and  peripheral  complexity  to  the  task  did  not  differentially  affect 


selection  times  in  the  case  of  click-open  or  walking  menus. 


CHAPTER  6 

ISOLATION  OF  THE  BUTTON-DEPRESSION  EFFECT 

Two  Motor  Hypotheses  About  the 
Button-Depression  Effect 

No  clear  evidence  was  found  for  either  the  central  memory-load 
hypothesis  or  the  peripheral  friction  hypothesis  in  Experiment  3.  We  now 
consider  two  additional  h3T)otheses  for  what  causes  the  slowness  in  moving 
the  mouse  while  the  button  is  depressed.  These  hypotheses  attribute  the 
slowness  to  neuromuscular  factors  and  are  presented  below: 

1)  The  difference  stems  from  increased  neuromuscvilar  complexity 
and  postural  awkwardness  when  moving  with  the  mouse  button  depressed. 

2)  The  difference  stems  from  a  greater  difficulty  of  coordinating  the 
end  of  the  arm  movement  with  a  button  release,  relative  to  coordinating  the 
movement  end  with  a  button  press. 

These  hypotheses  can  be  distinguished  by  which  portion  of  the 
movement  exhibits  the  slowness.  If  the  first  hypothesis  is  correct,  the 
difference  should  appear  throughout  the  main  phase  of  movement,  perhaps 
including  response  preparation.  If  the  second  hypothesis  is  correct,  then 
the  slowness  should  occur  at  the  end  of  the  menu  selection.  Here,  the 
difficulty  is  presumed  to  be  in  making  the  transition  from  moving  the 
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mouse  to  initiating  the  button  action  that  confirms  the  menu  selection. 

To  test  the  hypotheses,  new  analyses  of  the  data  from  Experiment  2 
were  conducted.  This  involved  parsing  the  movement  records  into  phases 
that  correspond  to  response  preparation,  mouse  movement,  and 
confirmation.  Here  response  preparation  is  measured  as  the  start-up 
phase  previously  defined.  Mouse  movement  and  confirmation  are 
measured  by  partitioning  the  first-level  execution  phase  of  the  movement 
into  two  segments;  mouse  movement  begins  with  intentional  movement 
and  ends  when  the  cursor  (and  the  mouse)  stops  moving;  confirmation 
begins  with  the  halt  of  the  cursor  and  finishes  with  the  button  action  that 
terminates  the  selection.  More  detailed  operational  definitions  for  the 
movement  phases  is  given  in  Appendix  B. 

New  Movement  Parsing  of  Single-Level  Selections 

As  in  the  previous  analyses  for  Experiment  2,  the  following 
ANOVA's  were  conducted  on  data  from  error-free  trials  from  blocks  two 
through  six.  These  analyses  differ  from  those  reported  thus  far  in  one 
respect.  Selections  of  the  first  menu  item  were  excluded.  The  rationale  for 
this  is  given  below. 

In  order  to  compare  hypotheses,  we  must  have  distinct  movement 
segments.  Examination  of  the  log  files  revealed  that  subjects  often  did  not 
move  the  mouse  when  selecting  the  first  menu  item.  This  happened 
because  it  was  often  possible  to  select  the  first  item  by  simply  pressing  (or 
releasing)  the  button.  Even  when  subjects  did  move,  the  distance  was  very 
small.  These  circumstances,  which  are  unique  to  the  first  menu  item, 
making  the  button  action  for  the  walking  menu  be  a  single  click  of  the 
mouse  button.  Similarly,  selecting  with  the  click-open  menu  is  changed  to 
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a  double  button-click.  As  a  consequence,  the  same  time  value  is  often 
attributed  to  the  start-up,  movement,  and  confirmation  phases  for  first-item 
selections.  This  confovmding  is  avoided  by  eliminating  the  first  menu 
position  from  the  analysis.  All  remaining  selections  have  an  unambiguous 
start-up  phase,  a  significant  movement  phase,  and  a  clear  confirmation 
phase  at  the  end. 


Results 

The  present  menu-style  manipulation  bears  directly  on  the 
hypotheses  being  tested  by  these  analyses.  Therefore,  for  the  sake  of  brevity, 
only  significant  results  involving  menu  style  are  reported  here.  None  of  the 
other  effects  give  any  new  insight  into  the  menu-selection  task. 

Start-Up  Phase 

The  effect  of  menu  style  almost  reached  significance  for  the  start-up 
phase  [F(l,7)  =5.0,  p  =  .06].  Click-open  selections  (58  msec)  were  31  msec 
faster  than  walking  menu  selections  (89  msec).  No  other  effects  involving 
menu  style  approached  significance  [all  p's  >.20]. 

Mouse-Movement  Phase 

Moving  the  mouse  was  57  msec  faster  for  the  click-open  (button-up) 
menus  than  for  the  walking  menus  [F(l,7)  =7.99,  p  >  .05].  Click-open 
averaged  650  msec  compared  to  707  msec  for  walking  menus.  No  other 
effects  involving  menu  style  approached  significance  [all  p's  >.20]. 

Confirmation  Phase 

There  was  virtually  no  effect  of  menu  style  on  the  time  for  the  final 


73 


button  action  to  confirm  the  menu  selection  [F(l,7)=0.05,  p  <  .80].  Click-open 
selections  (69  msec)  were  a  mere  3  msec  faster  than  walking-menu 
selections  (72  msec).  There  were,  however,  two  significant  interactions 
involving  menu  style.  The  interaction  between  menu  style  and  menu 
length  was  highly  significant,  with  the  walking  menu  being  much  more 
affected  by  menu  length  than  was  the  click-open  menu  [F(2, 14)=  16.28,  p  > 
.001].  The  three-way  interaction  involving  menu  style,  length,  and 
organization  [F(2,14)=12.07,  p  >  .001]  further  suggests  that  the  randomly- 
organized  walking  menus  have  the  greatest  increase  in  confirmation  time 
as  a  fimction  of  length. 


Discussion 

The  new  analyses  of  Experiment  2  finally  give  some  insight  into  the 
mechanism  for  the  slowness  of  moving  the  mouse  with  the  button 
depressed.  The  evidence  suggests  that  the  selection-time  difference  is 
caused  by  increased  postural  complexity  when  moving  with  the  mouse 
button  depressed.  The  difference  appears  in  the  mouse-movement  phase, 
which  suggests  that  moving  with  the  button  depressed  is  more  awkward 
than  moving  with  the  button  released. 

There  was  no  evidence  of  difficiilty  in  synchronizing  the  end  of  the 
mouse  movement  with  the  final  button  action.  Click-open  and  walking 
menus  yielded  the  same  duration  for  the  final  button  action. 

The  interaction  effects  involving  menu  style  and  menu  length  in  the 
confirmation  phase  are  intriguing.  The  disadvantage  of  selecting  from  a 
walking  menu  increases  with  the  number  of  items  on  the  menu.  This  is 
especially  true  for  the  randomly-organized  menus.  The  click-open 
confirmation  times  were  relatively  unaffected  by  the  length  and 
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organization  of  the  menus.  This  indicates  that  at  least  for  longer  menus, 
the  final  confirmation  phase  is  different  with  click-open  and  walking 
menus.  The  confirmation  phase  for  walking  menus  may  include  a  search 
of  the  entire  menu.  The  confirmation  phase  for  the  click-open  menu  may 
involve  a  local  search  which  is  not  influenced  by  the  length  or  organization 
of  the  menu. 


CHAPTER? 

GENERAL  DISCUSSION 


Summary  of  Major  Findings 

At  the  end  of  chapter  two,  several  questions  were  raised  about  mouse 
movements  and  menu  selection  in  HCI.  We  may  now  proceed  to  answer 
these  questions  as  follow: 

(1)  Is  the  speed  and  accuracy  of  menu  selection  sensitive  to  changes  in 
the  physical  actions  required  by  different  menu  styles?  If  so, 
which  aspects  of  selection  are  affected  by  the  difference? 

The  answer  to  the  first  part  of  this  question  is  unequivocally  yes. 
Selecting  from  click-open  menus  is  faster  and  more  accurate  than  selecting 
from  walking  menus.  The  difference  in  accuracy  appears  to  stem  from  the 
path  constraint  imposed  by  walking  menus.  Without  such  constraints  (e.g. 
one-level  menu  selections,  final-level  selections  in  multi-level  menus),  there 
is  no  appreciable  difference  in  accuracy  between  the  two  menu  styles.  The 
position  of  the  mouse  button  during  the  movement  contributes  to  the 
difference  in  speed.  Moving  the  mouse  with  the  button  depressed  is 
appreciably  slower  over  the  entire  movement  phase  than  moving  the  same 
distance  with  the  button  released. 

While  the  findings  about  total  selection  time  are  clear,  our  attempt  to 
isolate  the  effect  of  menu  style  in  specific  phases  of  the  movement  had 
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mixed  results.  In  two-level  menu  selection,  the  strongest  effects  appeared 
to  occur  during  the  second-level  selection  phase,  the  most  complex  phase  of 
movement.  Both  Experiments  1  and  3,  showed  that  the  pattern  of  results  for 
the  second-level  execution  phase  mimics  the  total  selection  times.  The 
start-up  time  was  also  influenced  by  the  menu  style  for  two-level  menu 
selections.  Initiating  movements  was  faster  with  click-open  menus  than 
with  walking  menus.  One-level  menu  selections  exhibited  the  same  trend, 
but  it  was  not  significant. 

Analyses  of  the  start-up  phase  failed  to  uncover  clear  dimensions  of 
response  complexity.  One  possible  dimension  mentioned  in  Chapter  2  is  the 
number  of  movement  segments  required  by  a  selection.  If  the  number  of 
movement  segments  is  a  dimension  of  response  complexity,  the  start-up 
time  should  be  shorter  for  one-level  selections  than  for  two-level  selections. 
However,  comparing  the  results  for  Experiment  2  (one-level  menus)  and 
Experiment  1  (two-level  menus)  reveals  that  the  start-up  time  increased 
slightly  for  walking  menus  (90  to  97  msec),  but  it  actually  decreased  for 
click-open  menus  (61  to  45  msec)  when  selecting  from  one  and  two-level 
menus,  respectively.  This  suggests  that  number  of  movement  segments 
may  not  be  a  complexity  dimension  for  menu  selection. 

Moreover,  the  first-level  execution  phase  is  somewhat  influenced  by 
menu  style.  For  one-level  selections,  the  trend  went  in  the  expected 
direction,  with  selections  from  click-open  menus  (682  msec)  taking  48  msec 
less  than  did  selections  from  walking  menus  (739  msec).  Further  parsing 
of  the  execution  phase  reveals  that  the  mouse-movement  phase,  not  the 
final  button  action,  mediated  the  observed  difference  between  the  two  menu 
styles. 
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(2)  How  are  the  applicability  and  parameter  values  of  Fitts'  law  affected 

by  the  subjects'  advance  knowledge  about  the  location  of  menu 
items? 

Fitts'  law  applies  quite  well  when  subjects  know  the  location  of  the 
menu  items  in  advance  and  do  not  have  to  search  extensively  for  them 
(Figure  3.5).  It  does  not  apply  when  substantial  visual  search  is  required  to 
locate  the  target  menu  item,  as  in  the  randomly-organized  menus.  A  linear 
ftmction,  based  on  the  serial  position  of  the  menu  item,  provides  the  best  fit 
for  the  randomly-organized  menus  (Figure  3.6). 

For  the  normally-organized  menus,  Fitts'  law  applies  equally  well  to 
both  menu  styles  (r^>  .95),  even  when  ID  is  calculated  in  terms  of  the 
vertical  distances  to  the  menu  items.  The  parameters  are  similar  for  the 
two  menu  styles  with  walking  menus  tending  to  yield  steeper  slopes  (Figure 

3.5) .  Fitts'  law  is  not  altered  by  the  niimber  of  items  in  a  menu  (Figure  4.4) 
or  the  munber  of  menu  levels  (Figure  4.3).  This  stability  bodes  well  for  the 
generalizability  of  the  present  resxilts  to  other  menu- selection  tasks. 

(3)  Is  visual  search  random  or  systematic  in  mouse-based  menu 

selection? 

There  is  clear  evidence  that  for  numeric  menus,  the  search  process 
is  systematic  when  the  menu  items  are  randomly-organized.  The  serial- 
position  fimctions  for  the  randomly-organized  menus  indicate  a  serial 
top-to-bottom  search  with  a  rate  of  approximately  125  msec  per  item  (Figure 

3.6) .  The  linearly  increasing  serial-position  function  also  indicates  that  the 
visual  search  is  predominantly  self-terminating.  However,  Experiment  2 
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showed  that  the  total  number  of  items  in  the  menu  influences  the  selection 
time  for  random  menus  (Figure  4.4),  whereas  a  strictly  self-terminating 
search  would  not  be  influenced  by  the  length  of  the  menu.  This  result 
suggests  that  there  is  an  additional  exhaustive  component  of  visual  search. 

On  the  other  hand,  for  the  normally-organized  menus,  the  evidence 
indicates  a  fast,  direct  search  process.  This  is  seen  in  the  relatively  fast 
selection  times  and  the  smaller  serial-position  effect  characterized  by  Fitts' 
law  for  movement  time.  The  lack  of  a  length  effect  in  normally-organized 
menus  suggests  that  here  the  search  is  either  self-terminating  or  faster 
than  other  concurrent  processes. 

(4)  Are  visual  search  and  movement  serial,  independent  processes,  as 
assumed  in  the  literature  on  menu  selection? 

The  interaction  between  menu  style  and  menu  organization  in 
Experiment  1  provides  evidence  that  the  search  and  movement  processes 
are  not  independent  (Figure  3.3),  Click-open  and  walking  menus  differ  only 
in  the  physical  actions  reqmred  to  make  the  selections.  Menu  organization 
(normal  or  random)  changes  only  the  amount  of  visual  search  required  to 
make  the  selections.  Therefore,  if  search  and  movement  were  serial 
independent  processes,  then  the  difference  in  selection  times  between 
normally  and  randomly-organized  menus  should  be  equal  for  the  two  menu 
styles  (Sternberg,  1969a).  However,  this  does  not  happen  here.  The  physical 
actions  associated  with  selecting  from  two-level  walking  menus  adversely 
affected  the  selection  times  for  randomly-organized  menus.  Examining  the 
amoimt  of  time  taken  right  before  the  final  button  action  for  one-level  menu 
selection,  suggests  that  an  extra  exhaustive  search  may  occur  at  the  end  of 
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the  selection  process  for  randomly-organized  walking  menus. 

(5)  Is  menu  selection  with  a  mouse  best  characterized  by  a  serial  or  a 
parallel  model? 

The  present  results  point  to  major  problems  with  the  Keystroke-Level 
Model.  It  overestimates  the  menu-selection  times  by  a  significant  amount. 
This  is  not  too  surprising  given  that  the  model  was  validated  originally  on 
naturalistic  tasks  and  is  meant  hold  for  a  wide  range  of  tasks.  More 
problematic  is  that  the  model’s  predictions  were  directly  opposite  of  the 
results  of  the  present  study. 

Part  of  the  problem  with  the  Keystroke-Level  Model  is  that  its 
parameters  do  not  embody  certain  salient  features  of  the  menu-selection 
task  (e.g.,  path  constraints  and  button  status  while  moving  the  mouse). 
Adding  more  parameters  to  the  model  should  improve  its  fit.  One  such 
parameter,  derived  from  the  present  studies,  is  the  added  time  for  moving 
with  the  mouse  button  depressed  (76  msec  per  menu  level).  Analysis  of  the 
final  button  action  in  one-level  menu  selection  adds  another  parameter 
value  of  70  msec  for  the  time  to  press  or  release  the  mouse  button. 

A  more  fundamental  problem  is  the  Keystroke-Level  Model's 
assumption  of  exclusively  serial  processing.  The  shape  of  the  serial- 
position  functions  show  that  adding  more  parameters  to  the  Keystroke- 
Level  Model  cannot  explain  the  data.  The  serial  position  fimctions  are 
better  explained  in  a  framework  that  provides  for  serial  and  concurrent 
processes,  as  under  the  Critical-Path  Model  (CPM).  Although  the 
experiments  in  this  dissertation  are  not  sufficient  to  fully  specify  such  a 
model,  they  do  provide  information  relevant  to  developing  one.  The 
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divergent  serial-position  curves  suggest  that  search  and  movement  are 
concurrent  processes.  To  be  specific,  one  variable  that  affects  which 
processes  are  on  the  critical  path  is  the  menu  organization.  The  two 
Critical-Path  Models  shown  in  Figure  7.1  illustrate  possible  relationships 
between  perceptual  and  motor  processes  for  a  one-level  menu  selection 
from  normal  and  randomly-organized  click-open  menus.  In  these  models, 
each  process  necessary  to  complete  the  task  is  represented  as  a  box  with  a 
duration  (e.g.,  a  saccade  takes  230  msec  (Olson  &  Olson,  1990).  The 
durations  for  searching  and  moving  are  taken  from  the  serial-position 
functions  of  Experiment  1.  The  numbers  in  the  boxes  represent  the 
positions  of  the  menu  items.  Sequential  dependencies  between  processes 
are  represented  by  lines  connecting  the  boxes  (e.g.,  the  final  button  press 
does  not  begin  until  both  the  eyes  and  the  cursor  have  been  moved  to  the 
target  menu  item).  There  are  no  lines  between  the  search  and  move  boxes. 
This  indicates  that  they  are  independent,  concurrent  processes.  The 
critical  path  (thick,  shaded  boxes)  is  the  set  of  processes  whose  duration 
contributes  to  the  total  task  time.  The  total  task  time  is  calculated  by 
summing  the  durations  of  the  processes  on  the  critical  path.  Processes  off 
the  critical  path  are  not  time  critical  and  have  flexibility  in  when  they  are 
executed. 

Figure  7.1  shows  that  the  critical  path  changes  substantially  as  a 
function  of  menu  organization.  For  normally-organized  menus,  the  mouse 
movement  determines  the  overall  time.  For  randomly-organized  menus, 
the  visual  search  determines  the  overall  time.  This  is  a  result  of  the 
parallel-processing  assumption  of  the  Critical-Path  Model.  It  explains  the 
shape  of  the  serial-position  curves  quite  well.  Identical  models  for  the 
walking  menus  can  be  formulated  by  replacing  the  values  for  the  move  and 
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search  processes  with  the  appropriate  values  from  the  walking-menu 
serial-position  functions. 


Figure  7.1  Critical-path  models  for  selecting  the  third  item 
from  normally  and  randomly  organized,  click-open  menus. 
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The  longer  times  for  the  first  serial  position  in  the  randomly- 
organized  menus  can  also  be  explained  by  looking  at  the  random- 
organization  CPM.  If  the  search  and  move  begin  at  the  same  time,  then 
after  the  search  of  item  one  is  completed,  the  cursor  will  have  already 
moved  to  item  two,  requiring  an  extra  movement  back  to  item  one  when  it  is 
the  target  menu  item.  This  prediction  is  supported  by  examining  the 
movement  trajectories  from  single-level  selections  in  Experiment  2.  In  the 
randomly-organized  menus,  40%  of  the  movements  to  first  menu  items 
were  overshoots,  compared  to  only  5%  overshoots  in  the  normal  menus. 

Of  course,  a  few  difficulties  may  arise  in  developing  a  CPM  model  for 
menu  selection.  It  is  not  clear  how  to  treat  a  process  whose  duration  is 
stochastic.  Extant  CPM  models  assume  all  processes  take  constant 
amoimts  of  time,  unlike  the  models  in  Figure  7.1.  It  is  also  not  clear  how  to 
handle  errors  and  error  recovery  with  a  CPM  model.  Finally,  determining 
the  placement  of  the  sequential  dependencies  is  not  obvious.  There  are 
several  alternative  models  that  lead  to  the  same  critical  path  as  in  Figure 
7.1.  Much  more  work  is  needed  therefore  to  generate  a  definitive  CPM 
model  for  menu  selection. 

Implications  for  Interface  Design 

It  is  a  large  step  from  a  laboratory  study  with  simple,  numeric 
menus  to  complex  real-world  interfaces.  Nevertheless,  the  findings  of  this 
dissertation,  do  have  implications  for  existing  and  emerging  human- 
computer  interfaces. 

The  implications  for  menu  selection  are  readily  apparent.  We  have 
found  that  click-open  menus  are  superior  to  walking  menus  on  measures 
of  speed  and  accuracy.  Moving  a  mouse  with  the  button  depressed  is  more 
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awkward  than  moving  it  with  the  button  released.  This  bears  on  the  Next 
computer  system,  which  provides  both  of  these  menu  styles.  The  present 
research  implies  that  with  experience,  Next  users  should  use  the  click-open 
menu  predominantly. 

The  implications  also  go  beyond  menu  selection.  Moving  the  mouse 
with  the  button  depressed  is  a  feature  of  many  tasks  on  current  systems. 

On  the  Macintosh,  for  example,  many  actions  (e.g.,  selecting,  moving, 
resizing,  animating)  performed  on  many  objects  (e.g.,  files,  icons,  windows, 
graphical  objects,  text  objects,  scroll  bars)  involve  moving  with  the  mouse 
button  depressed  (dragging).  While  the  time  costs  of  these  operations  may 
be  relatively  small,  as  in  the  current  study,  their  ubiqiaity  and  frequency 
means  that  replacing  them  with  clicking  operations,  where  appropriate, 
should  be  considered. 


Future  Directions 

There  are  three  future  directions  that  this  research  can  take.  One 
logical  next  step  would  be  to  investigate  the  other  main  motor  factor  found 
in  these  menu  styles,  path  constraint  during  movement.  We  can  pursue 
this  by  introducing  a  new,  hybrid  menu  style  into  the  investigation,  a 
button-up  walking  menu.  This  menu  style  resembles  walking  menus  with 
respect  to  how  submenus  are  accessed  by  moving  through  a  constrained 
"hot  zone",  but  it  is  similar  to  click-open  menus  in  that  the  movement 
occurs  with  the  mouse  button  released.  Comparing  the  button-up  walking 
menu  with  existing  menu  styles  will  provide  a  way  to  study  the  influence  of 
path  constraint  more  fully. 

Another  direction  for  future  research  is  to  understand  the  factors 
that  influence  the  suitability  of  Fitts’  law  in  various  HCI  tasks.  One 
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shortcoming  of  previous  work  on  HCI  is  the  over-generalization  of  Fitts' 
law  to  mouse  movements.  It  is  incorrect  to  state  that  Fitts'  law  applies  in 
general  to  mouse  movements,  and  to  then  use  that  statement  as  a  design 
guideline.  Current  studies  indicate  that  the  applicability  of  Fitts'  law 
depends  on  the  advance  information  that  a  user  has  about  the  location  of 
the  target. 

Even  when  Fitts'  law  does  apply,  as  for  normally-organized  menus, 
its  slope  and  intercept  depend  on  the  nature  of  the  selection  task.  This  is 
shown  in  Figure  7.2  which  outlines  Fitts'  law  for  three  computer-based 
selection  tasks  with  a  mouse. 

MOVEMENT  TARGETS 

Words  Embedded  In  Text 

(Card,  English,  &  Burr,  1978) 

Numeric  Walking  Menus  MT  =  329  +  207  ID,  r*=  0.99. 

(Current  Study,  Experiment  2.) 

Simple  Graphical  Targets  MT  =  108  +  392  ID,  r^  =  0.70. 

(Epps,  1986) 

Figure  7.2  Fitts'  law  for  three  computer-based  tasks. 

As  figure  7.2  indicates  there  is  a  systematic  change  in  both  the  slope 
and  the  intercept  across  these  three  tasks.  Two  factors  that  vary  across 
these  selection  tasks  might  account  for  the  difference  in  these  parameters. 
They  are  (1)  the  complexity  of  the  target,  and  (2)  the  context  in  which  it  is 
embedded. 

For  the  text-selection  task  (Card,  English,  &  Burr,  1978)  the  target  is 


EQUATION  FOR  FITTS'  LAW 
(all  times  in  msec) 

MT  =  1003  +  96  ID,  r*  =  0.83. 
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very  complex,  and  reading  is  required  as  part  of  recognizing  the  target.  The 
target  is  also  embedded  among  many  other  words  with  semantic  content. 

As  a  result,  Fitts'  law  has  a  relatively  large  intercept  and  a  shallow  slope. 
Its  form  may  stem  from  the  slower  cognitive  processes  reducing  the  impact 
of  motor  processes. 

At  the  other  extreme,  for  the  graphical-selection  task  (Epps,  1986),  the 
targets  are  simple  shapes,  with  little  semantic  content,  isolated  from  other 
possible  targets  on  the  screen.  Here  Fitts’  law  has  a  steep  slope  and  a  small 
intercept.  The  much  larger  effect  of  ID  on  selection  time  in  this  task  may 
occiur  because  of  the  isolation  of  the  targets,  the  lack  of  semantic  content,  or 
both. 

Relative  to  these  preceding  cases,  menu  selection  is  intermediate  both 
in  complexity  of  target  and  context.  In  menus,  the  set  of  command  names 
are  more  limited  than  in  written  text,  but  they  do  have  semantic  content. 

The  menu  items  are  imbedded  in  the  context  of  other  similar  targets  in  the 
same  area  of  the  screen.  The  slope  and  intercept  parameters  for  menu 
selection  are  between  those  for  embedded  text  selection  and  isolated 
graphical  selection. 

An  integrated  set  of  studies  that  manipulate  the  semantic  and  spatial 
context  of  aimed  movements  in  the  domain  of  HCI  will  provide  a  way  to 
specify  task  conditions  where  Fitts’  law  applies.  It  will  also  give  some 
indication  of  the  approximate  parameter  values  for  Fitts’  law,  based  on  a 
task  analysis  of  the  characteristics  that  affect  those  parameters. 

A  third  direction  that  future  research  must  take  is  to  make  the  menu 
selection  more  realistic  by  adding  more  cognitive  processing  to  the  task. 

This  can  be  accomplished  by  changing  the  menu  items  and  the  precues  in  a 
systematic  fashion.  Words  must  be  added  to  the  menus,  with  identical 
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precues,  as  in  the  current  studies.  Another  level  of  complexity  will  involve 
using  synonyms  or  class  inclusion  precues,  thus  requiring  subjects  to 
make  a  semantic  match  in  order  to  complete  the  selection.  Next  up  the 
ladder  of  realism  would  be  to  have  the  precues  represent  procedural  goals 
that  are  matched  with  the  menu  options.  It  will  be  interesting  to  see  how 
the  results  of  the  current  investigation  are  altered  by  these  suggested 
modifications  in  the  menu-selection  task.  Perhaps  some  of  the 
perceptual/motor  factors  will  continue  to  have  an  effect  in  the  presence  of 
more  cognitive  processes.  Also,  certain  factors  may  be  overwhelmed  by  the 
higher-order  processes. 

In  summary,  further  efforts  must  be  made  to  specify  processes  that 
are  important  for  real-life  menu  selection.  This  dissertation  serves  as  an 
existence  proof  of  concurrent  information-processing  in  HCI  tasks.  It 
remains  to  be  seen  whether  the  concurrent  nature  of  processing  in  the 
menu- selection  task  will  generalize  to  other,  more  realistic,  menu-selection 
tasks. 


APPENDIX  A 


COMPUTER  PROGRAM  FOR  DATA  COLLECTION 


A  computer  program  was  developed  to  run  the  experimental  sessions 
and  collect  the  data.  It  is  called  the  Mice  and  Menus  Program  (MMP),  and 
rims  on  an  IBM  AT  compatible  computer  with  a  Microsoft  mouse  as  the 
selection  device.  MMP  presents  menus  on  the  screen  of  the  computer, 
tracks  the  movement  of  the  cursor  to  select  from  the  menus,  and  collects 
fine-grained  timing  and  position  data. 

The  program  allows  the  experimenter  to  name  and  organize  the 
items  in  a  hierarchical  menu  ranging  from  one  to  three  menu  levels,  with 
one  to  nine  items  at  each  menu  level.  The  experimenter  can  also  control 
which  menu  items  are  to  be  selected  in  an  experimental  session  and  how 
many  practice  trials  are  included  at  the  beginning  of  each  block  of  trials. 

The  appearance  of  the  menus  can  be  manipulated  by  specifying  the 
width  and  height  of  the  menu  items,  the  ordering  of  the  menu  items  within 
a  menu,  and  whether  or  not  non-target  items  appear  on  the  menu.  The 
actions  for  using  the  menu  can  also  be  controlled  by  choosing  one  of  the 
supported  menu  styles  (currently  click-open,  walking,  and  button-up 
walking). 

MMP  generates  three  data  files  for  each  subject.  A  short  data  file 
(appended  with  ".x")  shows  information  at  the  block  level  including  the 
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number  of  practice  trials,  the  average  correct  selection  time  for  the  block  of 
trials,  the  number  incorrect  trials,  and  the  name  of  the  file  that  contains  the 
information  about  the  characteristics  of  that  block.  A  second  data  file 
(appended  with  ".o")  gives  information  for  each  trial,  including  the  style  of 
menu,  the  target  menu  items,  the  menu  items  selected,  and  the  timing  and 
position  of  the  total  selection  time.  The  third  long  data  file  (appended  with 
".1")  contains  the  same  information  as  the  second  data  file  along  with  a 
record  of  the  cursor  position,  mouse-button  status,  and  elapsed  time  from 
the  beginning  of  the  trial.  The  timing  resolution  of  the  data  files  are  5  msec. 


APPENDIX  B 


OPERATIONAL  DEFINITIONS  FOR  MOVEMENT 
PHASES  USED  IN  THE  EXPERIMENTS 

Many  of  the  analyses  in  this  dissertation  are  based  on  segmenting 
the  total  selection  time  into  various  movement  phases  (e.g.,  start-up,  first- 
level  execution,  second-level  selection,  mouse  movement,  confirmation). 
This  appendix  will  give  detailed  operational  definitions  for  the  movement 
phases  as  well  as  describing  the  parsing  algorithms  that  were  used  to 
extract  the  times  from  the  data  files  produced  by  MMP  (the  data  collection 
program  described  in  Appendix  A). 

The  long  data  files  (appended  with  ".1")  were  used  to  identify  the 
movement  phases.  These  files  contained  a  record  of  the  cursor  position  in  x 
and  y  coordinates,  mouse-button  status  (up  or  down),  and  elapsed  time  from 
the  beginning  of  the  trial.  This  information  was  obtained  by  using  an 
interrupt  provided  by  the  Microsoft  Mouse  software  {Microsoft  Mouse 
Programmer's  Reference,  1989)  that  reported  the  information  whenever  a 
"mouse  event"  occurred.  A  "mouse  event"  is  defined  as  a  change  in  the 
position  of  the  mouse  of  at  least  one  "Mickey"  (approximately  1/200  of  an 
inch),  or  a  change  in  the  status  of  the  mouse  button.  The  interrupt  is 
strictly  event-driven,  only  reporting  when  a  "mouse  event"  occurs.  The 
maximum  resolution  of  the  interrupt  is  5  msec.  Any  events  that  happen  in 
a  5  msec  interval  are  reported  as  a  single  entry  in  the  data  file. 
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Two-Level  Menu  Selection 

Explanation  of  Phase-Parsing  Graphs 

In  Experiment  1  and  3  subjects  selected  from  two-level  menus.  The 
total  selection  time  was  divided  into  three  movement  phases:  start-up, 
first-level  execution,  and  second-level  selection.  Figure  B.l  shows  how  these 
movement  phases  were  determined  from  sample  trials  using  click-open 
and  walking  menus.  The  x-axis  on  these  graphs  represent  the  elapsed  time 
(in  msec)  for  a  trial.  The  zero  point  is  determined  when  the  first  menu-level 
appears  on  the  screen.  Each  tick  mark  represents  40  msec.  The  y-axis 
represents  the  movement  of  the  cursor. 

While  the  data  file  records  the  absolute  position  of  the  on-screen 
cursor  in  x  and  y  coordinates,  it  is  difficult  to  represent  both  these 
dimensions  and  time  in  a  two-dimensional  graph.  Also,  the  parsing 
algorithms  are  not  dependent  on  the  direction  of  movement,  but  rather  on 
the  detection  of  movement  in  any  direction,  changes  in  mouse  button  status, 
and  movement  through  specific  screen  coordinates.  Therefore,  these 
graphs  increment  the  value  on  the  y-axis  for  every  time  period  that  anv 

in  position  is  detected.  The  "Change  in  Position"  axis  is  an  ordinal 
scale  and  does  not  represent  the  distance  or  the  direction  of  the  movement. 

It  IS,  however,  highly  correlated  with  the  principal  direction  of  movement. 
Since  it  differentiates  between  periods  of  rapid  movement  and  pauses  in  the 
movement  trajectory,  it  is  sufficient  for  our  needs  to  parse  the  interval  into 
segments. 

Each  box  on  the  graphs  corresponds  to  a  change  in  either  mouse 
position  or  button  status  that  is  detected  by  the  data  collection  software.  The 
connecting  lines  show  the  time  delay  between  events.  A  slightly  sloping 
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Figure  B.l  Examples  of  Movement ! 
Two-Level  Menu  Selections  (Experii 
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line  between  boxes  of  the  same  color  represents  a  pause  in  the  movement, 
followed  by  a  change  in  position.  A  horizontal  line  between  boxes  of 
different  colors  represents  a  change  in  button  status  with  no  movement  of 
the  mouse.  Finally,  the  vertical  dashed  lines,  show  the  demarcations 
between  the  various  movement  phases.  The  rationale  for  determining  these 
phases  will  be  described  in  the  next  section. 

Parsing  Rationale  for  Movement  Phases 

Start-Up  Phase 

The  operational  definition  for  the  start-up  phase  is  identical  for  both 
click-open  and  walking  menus  in  all  of  the  analyses  in  this  dissertation.  It 
begins  following  the  action  to  bring  up  the  first-level  menu  (button  click  for 
click-open,  and  button  press  for  walking),  and  ends  when  intentional 
movement  of  the  mouse  begins.  A  majority  of  the  time,  the  ending  point  is 
the  very  first  entry  in  the  data  file  as  shown  in  the  click-open  parsing 
example  (top  graph  in  Figure  B.l). 

Occasionally  the  first  entry  is  very  fast  and  followed  by  a  significant 
pause  before  the  next  entry  as  seen  in  the  walking  parsing  example  (bottom 
graph  in  Figure  B.l).  This  is  caused  by  muscle  tremor,  or  unintentional 
mouse  movement  which  accompanies  the  pressing  of  the  mouse  button.  To 
overcome  this,  the  movement  start  is  operationally  defined  as  the  first  of 
three  consecutive  movements,  each  of  which  have  a  duration  of  less  than  40 
msec.  This  ensures  that  the  movement  is  sustained  and  intentional.  In  the 
example  at  the  bottom  of  Figure  B.l,  the  first  entry  is  considered  to  be  in  the 
start-up  phase  since  it  is  followed  by  a  pause  greater  than  40  msec.  The 
second  entry  is  identified  as  the  end  of  the  start-up  phase  since  the  two 
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following  entries  occur  in  rapid  succession. 

First— Level  Execution  Phase 

This  movement  phase  commences  when  the  subject  begins 
intentional  movement  of  the  mouse,  and  ends  when  the  second  menu  level 
is  visible  on  the  screen.  The  action  which  marks  the  end  of  this  movement 
phase  is  different  for  the  two  menu  styles.  For  click-open  menus,  the 
termination  point  is  a  button  press  anywhere  in  the  target  menu  item,  as 
shown  in  Figure  B.l.  For  walking  menus,  it  is  signalled  by  moving  the 
cursor  into  the  "hot  zone"  region  at  the  right  of  the  target  menu  item.  For 
these  experiments  the  edge  of  the  "hot  zone"  has  an  x  coordinate  of  51.  The 
first  entry  in  the  data  file  which  is  in  the  target  menu  item  and  has  an  x 
coordinate  of  51  or  greater  is  specified  as  the  end  of  the  first-level  execution 
phase. 

Second-Level  Selection  Phase 

This  movement  phase  begins  with  the  appearance  of  the  second-level 
menu,  and  ends  with  the  action  that  completes  the  menu-selection  and 
causes  the  menu  to  disappear  (button  press  for  click-open,  and  button 
release  for  walking).  The  end  of  this  movement  phase  is  always  the  last 
entry  in  the  data  file  for  that  trial. 

Single-Level  Menu  Selection 

Experiment  2  subjects  selected  from  single-level  menus.  Figure  B.2 
shows  how  these  movement  phases  were  determined  from  sample  trials 
using  click-open  and  walking  menus.  These  graphs  are  identical  in  form 
to  those  in  Figure  B.l.  The  determination  of  the  start-up  phase  is  also  the 
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same  as  for  two-level  selections.  The  first-level  execution  phase,  however,  is 
different  for  single-level  selections.  This  movement  phase  commences 
when  the  subject  begins  intentional  movement  of  the  mouse,  and  ends 
when  a  menu  item  is  selected,  causing  the  menu  to  disappear.  The  action 
which  ends  the  selection  is  a  button  press  anywhere  in  the  menu  item  for 
the  click-open  menus,  and  a  button  release  anywhere  in  the  menu  item  for 
the  walking  menus.  This  action  is  always  the  last  entry  in  the  data  file  for  a 
particular  trial,  as  shown  in  Figure  B.2. 

New  Movement  Parsing  for  Single-Level  Selections 

In  order  to  discriminate  between  two  motor  hypotheses  for  the  button- 
depression  effect,  it  was  necessary  to  obtain  a  more  detailed  parsing  of  the 
single-level  menu  selections  from  Experiment  2.  This  was  accomplished  by 
taking  the  first-level  execution  phase,  described  in  the  previous  section,  and 
dividing  it  into  two  movement  segments;  the  mouse  movement  phase,  and 
the  confirmation  phase.  This  new  movement  parsing  is  shown  in  Figure 
B.3. 

Here,  the  start-up  phase  is  defined  as  in  the  previous  analyses.  The 
mouse-movement  phase  is  identical  for  both  click-open  and  walking 
menus.  It  begins  with  intentional  movement  and  ends  when  the  mouse 
movement  ceases  immediately  before  the  final  button  action.  This  is  always 
the  second  to  the  last  entry  in  the  data  file  and  has  the  same  x-y  coordinates 
as  the  final  button  action.  The  confirmation  phase  is  the  time  between  the 
last  mouse  movement  and  the  final  button  action  (button  press  for  click- 
open,  and  button  release  for  walking).  This  is  always  the  difference  between 
the  last  and  the  second  to  last  entry  in  the  data  file. 
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